2024-05-10T20:41:02.187 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T20:41:02.187 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_vshankar@teuthology --name vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} -- /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/orig.config.yaml 2024-05-10T20:41:02.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T20:41:02.354 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344 branch: wip-vshankar-testing-20240509.114851-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7701344' kernel: client: branch: testing flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240509.114851-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 7b710596db3f44de0a4b72eff459db857656fe05 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 7b710596db3f44de0a4b72eff459db857656fe05 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240509.114851-debug sha1: 7b710596db3f44de0a4b72eff459db857656fe05 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2119 sha1: 7b710596db3f44de0a4b72eff459db857656fe05 sleep_before_teardown: 0 subset: 618/1024 suite: fs suite_branch: wip-vshankar-testing-20240509.114851-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7b710596db3f44de0a4b72eff459db857656fe05 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKAu78QdBJQI4w+tcTCL0d1bjZHGzNS50g5kx2KBlyg8TK+G0u/Pw6hFBFp5E/iYH87HbHv4ENdYHxGRxpz+VeI= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHrcME+lzeCYOa/XtrRHQXNNS8PdEIgWMGVyxcYoXII00Vk+jDM2dmFb0lYaZNsImSm5cGbWSH4p66sBLNYVilc= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0ULr/U/puljGF5t830NJviseco3C4f1/FfF0S1C8RBn/cvY7MLBqLD0njoYk49fStnogMv3c3MnKYvXVKxmv4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_15:47:15 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T20:41:02.354 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa; will attempt to use it 2024-05-10T20:41:02.355 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks 2024-05-10T20:41:02.355 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T20:41:02.356 INFO:teuthology.task.internal:Checking packages... 2024-05-10T20:41:02.381 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7b710596db3f44de0a4b72eff459db857656fe05' 2024-05-10T20:41:02.381 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T20:41:02.381 INFO:teuthology.packaging:ref: None 2024-05-10T20:41:02.381 INFO:teuthology.packaging:tag: None 2024-05-10T20:41:02.381 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240509.114851-debug 2024-05-10T20:41:02.381 INFO:teuthology.packaging:sha1: 7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:41:02.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240509.114851-debug 2024-05-10T20:41:02.692 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3590.g7b710596 2024-05-10T20:41:02.694 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T20:41:02.700 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T20:41:02.700 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T20:41:02.707 INFO:teuthology.task.internal:Saving configuration 2024-05-10T20:41:02.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T20:41:02.736 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T20:41:02.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.598082', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKAu78QdBJQI4w+tcTCL0d1bjZHGzNS50g5kx2KBlyg8TK+G0u/Pw6hFBFp5E/iYH87HbHv4ENdYHxGRxpz+VeI='} 2024-05-10T20:41:02.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.597126', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHrcME+lzeCYOa/XtrRHQXNNS8PdEIgWMGVyxcYoXII00Vk+jDM2dmFb0lYaZNsImSm5cGbWSH4p66sBLNYVilc='} 2024-05-10T20:41:02.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.598859', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0ULr/U/puljGF5t830NJviseco3C4f1/FfF0S1C8RBn/cvY7MLBqLD0njoYk49fStnogMv3c3MnKYvXVKxmv4='} 2024-05-10T20:41:02.805 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T20:41:02.811 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T20:41:02.811 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T20:41:02.811 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T20:41:02.812 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T20:41:02.935 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff8c7da60d0>, signals=[15]) 2024-05-10T20:41:02.935 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T20:41:02.941 INFO:teuthology.task.internal:Opening connections... 2024-05-10T20:41:02.941 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T20:41:02.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:41:03.025 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-05-10T20:41:03.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:41:03.100 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-05-10T20:41:03.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:41:03.174 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T20:41:03.180 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-05-10T20:41:03.199 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-05-10T20:41:03.199 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-05-10T20:41:03.257 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T20:41:03.258 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T20:41:03.259 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-05-10T20:41:03.276 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-05-10T20:41:03.294 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-05-10T20:41:03.295 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-05-10T20:41:03.350 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-05-10T20:41:03.351 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T20:41:03.352 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T20:41:03.352 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T20:41:03.352 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-05-10T20:41:03.376 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-05-10T20:41:03.394 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-05-10T20:41:03.395 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T20:41:03.452 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-05-10T20:41:03.453 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T20:41:03.453 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T20:41:03.453 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T20:41:03.453 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-05-10T20:41:03.476 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T20:41:03.487 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T20:41:03.493 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T20:41:03.493 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-05-10T20:41:03.495 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-05-10T20:41:03.498 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-05-10T20:41:03.513 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T20:41:03.520 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T20:41:03.520 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-05-10T20:41:03.554 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-05-10T20:41:03.556 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-05-10T20:41:03.692 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T20:41:03.770 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T20:41:03.785 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-10T20:41:03.785 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-10T20:41:03.785 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-10T20:41:03.786 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T20:41:03.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:41:03.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-10T20:41:04.131 DEBUG:teuthology.task.kernel:sha1 for client.0 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T20:41:04.131 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-10T20:41:04.132 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-10T20:41:04.150 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-437.el9.x86_64 2024-05-10T20:41:04.150 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T20:41:04.151 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-10T20:41:04.151 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum upgrade -y linux-firmware 2024-05-10T20:41:06.826 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.1 MB 00:01 2024-05-10T20:41:09.527 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-10T20:41:15.003 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-05-10T20:41:16.504 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-05-10T20:41:17.864 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-10T20:41:23.164 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-10T20:41:25.332 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-10T20:41:25.333 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-10T20:41:25.334 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-10T20:41:25.438 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T20:41:25.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T20:41:25.587 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-10T20:41:25.587 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-10T20:41:25.590 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi050.front.sepia.ceph.com to complete... 2024-05-10T20:41:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10 20:41:59 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-10T20:41:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:FINISHED --2024-05-10 20:41:59-- 2024-05-10T20:41:59.459 INFO:teuthology.orchestra.run.smithi050.stderr:Total wall clock time: 34s 2024-05-10T20:41:59.460 INFO:teuthology.orchestra.run.smithi050.stderr:Downloaded: 1 files, 341M in 33s (10.3 MB/s) 2024-05-10T20:41:59.463 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T20:41:59.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-10T20:42:01.647 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying... ######################################## 2024-05-10T20:42:02.152 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing... ######################################## 2024-05-10T20:42:02.264 INFO:teuthology.orchestra.run.smithi050.stdout:Updating / installing... 2024-05-10T20:42:10.631 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-10T20:42:48.732 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-10T20:42:48.733 DEBUG:teuthology.orchestra.run.smithi050:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-10T20:42:48.807 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.807 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-10T20:42:48.808 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-10T20:42:48.809 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-10T20:42:48.810 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-10T20:42:48.811 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-10T20:42:48.812 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-10T20:42:48.813 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-10T20:42:48.814 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-10T20:42:48.815 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-10T20:42:48.816 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-10T20:42:48.817 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-10T20:42:48.818 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-10T20:42:48.819 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-10T20:42:48.820 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-10T20:42:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-10T20:42:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-10T20:42:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-10T20:42:48.824 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-10T20:42:48.825 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-10T20:42:48.826 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-10T20:42:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-10T20:42:48.828 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-10T20:42:48.829 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-10T20:42:48.830 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-10T20:42:48.831 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-10T20:42:48.832 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-10T20:42:48.833 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-10T20:42:48.834 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-10T20:42:48.835 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-10T20:42:48.836 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-10T20:42:48.837 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-10T20:42:48.838 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-10T20:42:48.839 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-10T20:42:48.840 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-10T20:42:48.841 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-10T20:42:48.842 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-10T20:42:48.843 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-10T20:42:48.844 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-10T20:42:48.845 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-10T20:42:48.846 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-10T20:42:48.847 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-10T20:42:48.848 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-10T20:42:48.849 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-10T20:42:48.850 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-10T20:42:48.851 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-10T20:42:48.852 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-10T20:42:48.853 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-10T20:42:48.854 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-10T20:42:48.855 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-10T20:42:48.855 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-10T20:42:48.855 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-10T20:42:48.859 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-10T20:42:48.859 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-10T20:42:48.859 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-10T20:42:48.859 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-10T20:42:48.860 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-10T20:42:48.861 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-10T20:42:48.862 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-10T20:42:48.863 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-10T20:42:48.864 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-10T20:42:48.865 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-10T20:42:48.866 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-10T20:42:48.867 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-10T20:42:48.868 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-10T20:42:48.869 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-10T20:42:48.870 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-10T20:42:48.871 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-10T20:42:48.872 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-10T20:42:48.873 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-10T20:42:48.874 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-10T20:42:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-10T20:42:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-10T20:42:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-10T20:42:48.878 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-10T20:42:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-10T20:42:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-10T20:42:48.881 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-10T20:42:48.882 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-10T20:42:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-10T20:42:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-10T20:42:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-10T20:42:48.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-10T20:42:48.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-10T20:42:48.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-10T20:42:48.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-10T20:42:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-10T20:42:48.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-10T20:42:48.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-10T20:42:48.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-10T20:42:48.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-10T20:42:48.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-10T20:42:48.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-10T20:42:48.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-10T20:42:48.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-10T20:42:48.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-10T20:42:48.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-10T20:42:48.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-10T20:42:48.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-10T20:42:48.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-10T20:42:48.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-10T20:42:48.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-10T20:42:48.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-10T20:42:48.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-10T20:42:48.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-10T20:42:48.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-10T20:42:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-10T20:42:48.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-10T20:42:48.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-10T20:42:48.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-10T20:42:48.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-10T20:42:48.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-10T20:42:48.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-10T20:42:48.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-10T20:42:48.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-10T20:42:48.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-10T20:42:48.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-10T20:42:48.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-10T20:42:48.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-10T20:42:48.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-10T20:42:48.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-10T20:42:48.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-10T20:42:48.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-10T20:42:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-10T20:42:48.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-10T20:42:48.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-10T20:42:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-10T20:42:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-10T20:42:48.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-10T20:42:48.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-10T20:42:48.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-10T20:42:48.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-10T20:42:48.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-10T20:42:48.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-10T20:42:48.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-10T20:42:48.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-10T20:42:48.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-10T20:42:48.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-10T20:42:48.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-10T20:42:48.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-10T20:42:48.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-10T20:42:48.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-10T20:42:48.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-10T20:42:48.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-10T20:42:48.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-10T20:42:48.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-10T20:42:48.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-10T20:42:48.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-10T20:42:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-10T20:42:48.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-10T20:42:48.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T20:42:48.958 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.959 DEBUG:teuthology.orchestra.run.smithi050:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-10T20:42:48.983 INFO:teuthology.orchestra.run.smithi050.stdout:postinstall scriptlet (using /bin/sh): 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout:for file in vmlinuz System.map config; do 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: fi 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout:done 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-10T20:42:48.984 INFO:teuthology.orchestra.run.smithi050.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:48.985 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-05-10T20:42:48.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T20:42:49.061 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-05-10T20:42:49.062 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:42:49.063 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T20:42:49.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T20:42:49.064 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:49.064 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T20:42:49.597 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-05-10T20:42:51.026 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T20:42:51.062 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-05-10T20:42:51.064 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-05-10T20:42:51.095 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.iQZ7Cma5mB 2024-05-10T20:42:51.096 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iQZ7Cma5mB 2024-05-10T20:42:51.162 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.iQZ7Cma5mB 2024-05-10T20:42:51.309 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.iQZ7Cma5mB is 6KB 2024-05-10T20:42:51.328 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.iQZ7Cma5mB 2024-05-10T20:42:51.343 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T20:42:51.414 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T20:42:51.414 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T20:42:51.414 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T20:42:51.414 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-10T20:42:51.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:42:51.574 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-05-10T20:43:21.577 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-10T20:43:21.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:43:21.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:43:39.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-05-10T20:43:48.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:43:48.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:44:48.940 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T20:45:00.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:45:00.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:45:01.243 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-10T20:45:01.883 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:45:01.883 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-10T20:45:01.883 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-10T20:45:01.938 INFO:teuthology.orchestra.run.smithi050.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:45:01.939 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-10T20:45:01.939 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T20:45:01.939 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T20:45:02.940 DEBUG:teuthology.parallel:result is None 2024-05-10T20:45:02.940 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T20:45:02.950 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T20:45:02.950 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T20:45:02.957 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T20:45:02.960 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T20:45:03.011 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T20:45:03.022 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T20:45:03.035 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T20:45:03.036 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T20:45:03.043 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T20:45:03.045 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T20:45:03.115 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T20:45:03.126 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T20:45:03.127 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T20:45:03.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:45:03.143 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T20:45:03.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:45:03.159 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T20:45:03.167 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T20:45:03.185 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T20:45:03.227 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.237 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.252 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.272 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.290 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.352 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T20:45:03.357 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T20:45:03.366 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T20:45:03.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T20:45:03.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T20:45:03.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T20:45:03.459 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T20:45:03.469 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T20:45:03.469 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T20:45:03.474 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T20:45:03.476 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T20:45:03.523 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:03.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:03.706 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:03.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:03.882 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:45:03.882 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T20:45:03.969 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:04.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:04.063 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:04.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:04.187 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:45:04.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T20:45:04.252 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:04.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T20:45:04.345 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:04.427 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T20:45:04.471 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:45:04.471 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T20:45:04.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-05-10T20:45:04.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-05-10T20:45:04.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-05-10T20:45:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T20:45:04.613 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T20:45:04.652 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T20:45:05.013 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T20:45:05.020 INFO:teuthology.task.internal:Starting timer... 2024-05-10T20:45:05.020 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T20:45:05.033 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T20:45:05.042 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-10T20:45:05.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-05-10T20:45:05.139 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-05-10T20:45:05.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-05-10T20:45:05.198 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-05-10T20:45:05.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-05-10T20:45:05.270 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-05-10T20:45:05.272 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T20:45:05.273 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-05-10T20:45:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-05-10T20:45:05.337 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-05-10T20:45:05.357 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-05-10T20:45:05.358 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-05-10T20:45:05.384 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-05-10T20:45:05.385 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-05-10T20:45:05.385 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T20:45:05.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:45:05.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T20:45:05.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:45:05.491 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T20:45:05.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T20:45:05.517 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T20:45:05.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-05-10T20:45:05.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-05-10T20:45:05.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-05-10T20:45:05.639 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T20:45:05.648 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-10T20:45:05.653 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-10T20:45:05.780 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T20:45:05.797 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T20:45:05.799 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T20:50:52.217 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-05-10T20:50:52.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:50:52.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:50:52.341 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-10T20:50:52.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-10T20:50:52.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-10T20:50:52.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:50:52.572 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-10T20:50:52.654 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-10T20:50:52.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-10T20:50:52.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T20:50:52.729 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-05-10T20:50:52.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-10T20:50:52.813 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T20:50:52.824 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T20:50:52.824 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T20:50:52.825 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T20:50:52.833 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T20:50:52.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T20:50:52.836 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T20:50:52.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T20:50:52.864 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T20:50:52.877 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T20:50:52.880 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T20:50:52.894 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T20:50:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-05-10T20:50:52.912 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-05-10T20:50:52.916 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-05-10T20:50:52.925 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T20:50:52.926 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-05-10T20:50:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T20:50:52.938 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T20:50:52.939 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T20:50:52.952 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T20:50:52.972 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T20:50:52.992 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:52.995 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.009 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-05-10T20:50:53.012 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T20:50:53.013 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-10T20:50:53.013 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.013 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.013 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.013 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-05-10T20:50:53.086 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-05-10T20:50:53.126 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T20:50:53.168 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T20:50:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.329 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T20:50:53.331 INFO:teuthology.run_tasks:Running task install... 2024-05-10T20:50:53.339 DEBUG:teuthology.task.install:project ceph 2024-05-10T20:50:53.339 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05'}} 2024-05-10T20:50:53.339 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05'} 2024-05-10T20:50:53.340 INFO:teuthology.task.install:Using flavor: default 2024-05-10T20:50:53.346 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T20:50:53.346 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T20:50:53.346 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'tag': None, 'wait_for_package': False} 2024-05-10T20:50:53.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:50:53.351 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'tag': None, 'wait_for_package': False} 2024-05-10T20:50:53.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:50:53.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'tag': None, 'wait_for_package': False} 2024-05-10T20:50:53.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:50:53.499 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/ 2024-05-10T20:50:53.499 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3590.g7b710596 2024-05-10T20:50:53.639 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/ 2024-05-10T20:50:53.639 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3590.g7b710596 2024-05-10T20:50:53.643 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/ 2024-05-10T20:50:53.644 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3590.g7b710596 2024-05-10T20:50:53.741 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T20:50:53.741 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:50:53.741 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T20:50:53.748 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T20:50:53.748 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:50:53.748 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T20:50:53.768 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T20:50:53.768 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b710596db3f44de0a4b72eff459db857656fe05/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T20:50:53.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T20:50:53.837 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T20:50:53.838 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b710596db3f44de0a4b72eff459db857656fe05/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T20:50:53.861 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T20:50:53.862 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:50:53.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T20:50:53.890 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T20:50:53.890 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b710596db3f44de0a4b72eff459db857656fe05/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T20:50:53.908 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T20:50:53.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T20:50:53.972 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-05-10T20:50:53.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T20:50:53.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-05-10T20:50:54.032 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T20:50:54.088 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T20:50:54.095 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-05-10T20:50:54.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-05-10T20:50:54.176 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-05-10T20:50:54.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-05-10T20:50:54.201 INFO:teuthology.orchestra.run.smithi142.stdout:54 files removed 2024-05-10T20:50:54.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T20:50:54.323 INFO:teuthology.orchestra.run.smithi092.stdout:54 files removed 2024-05-10T20:50:54.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T20:50:54.525 INFO:teuthology.orchestra.run.smithi050.stdout:54 files removed 2024-05-10T20:50:54.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T20:50:54.959 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 177 kB/s | 84 kB 00:00 2024-05-10T20:50:55.077 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-05-10T20:50:55.326 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-10T20:50:55.476 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 179 kB/s | 84 kB 00:00 2024-05-10T20:50:55.519 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-10T20:50:55.694 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-05-10T20:50:55.895 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-10T20:50:56.077 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-05-10T20:50:56.578 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-10T20:50:56.643 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.1 MB 00:00 2024-05-10T20:50:57.201 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-10T20:50:57.336 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.1 MB 00:01 2024-05-10T20:50:58.303 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 33 MB/s | 19 MB 00:00 2024-05-10T20:50:59.138 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-10T20:51:01.203 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-05-10T20:51:02.404 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-10T20:51:03.472 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.2 MB 00:00 2024-05-10T20:51:04.021 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-10T20:51:05.056 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-10T20:51:05.600 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-10T20:51:07.093 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-10T20:51:10.781 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-10T20:51:12.298 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 9.0 kB/s | 1.7 kB 00:00 2024-05-10T20:51:13.058 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:13.058 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:13.069 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T20:51:13.070 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T20:51:13.071 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T20:51:13.072 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T20:51:13.073 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T20:51:13.076 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T20:51:13.077 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T20:51:13.139 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T20:51:13.156 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.0.0-3590.g7b710596.el9 ceph 6.6 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.4 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 863 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3590.g7b710596.el9 ceph 147 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.0.0-3590.g7b710596.el9 ceph 1.5 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 142 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 3.5 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 7.4 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 49 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 10 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.0.0-3590.g7b710596.el9 ceph 43 M 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 273 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 764 k 2024-05-10T20:51:13.157 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 224 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.0.0-3590.g7b710596.el9 ceph 27 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 18 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 32 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 756 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 127 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T20:51:13.158 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.0.0-3590.g7b710596.el9 ceph 162 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.0.0-3590.g7b710596.el9 ceph 321 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 300 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 99 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 86 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.0 M 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 169 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T20:51:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.3 M 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.1 M 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 20 M 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 34 k 2024-05-10T20:51:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.0.0-3590.g7b710596.el9 ceph 2.3 M 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 274 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.0.0-3590.g7b710596.el9 ceph 4.9 M 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 18 M 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 16 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.0.0-3590.g7b710596.el9 ceph 25 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T20:51:13.161 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.0.0-3590.g7b710596.el9 ceph 157 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 497 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.2 M 2024-05-10T20:51:13.162 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T20:51:13.163 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 45 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 135 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T20:51:13.164 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T20:51:13.165 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T20:51:13.166 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T20:51:13.167 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T20:51:13.168 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 8 Packages 2024-05-10T20:51:13.169 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:51:13.170 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 219 M 2024-05-10T20:51:13.170 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-05-10T20:51:14.032 INFO:teuthology.orchestra.run.smithi142.stdout:(1/156): ceph-19.0.0-3590.g7b710596.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-10T20:51:14.202 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 711 kB/s | 6.2 MB 00:08 2024-05-10T20:51:14.399 INFO:teuthology.orchestra.run.smithi142.stdout:(2/156): ceph-fuse-19.0.0-3590.g7b710596.el9.x8 2.3 MB/s | 863 kB 00:00 2024-05-10T20:51:14.474 INFO:teuthology.orchestra.run.smithi142.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.9 MB/s | 147 kB 00:00 2024-05-10T20:51:14.567 INFO:teuthology.orchestra.run.smithi142.stdout:(4/156): ceph-base-19.0.0-3590.g7b710596.el9.x8 7.0 MB/s | 5.4 MB 00:00 2024-05-10T20:51:14.583 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:14.584 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:14.594 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T20:51:14.595 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T20:51:14.596 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T20:51:14.597 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T20:51:14.598 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T20:51:14.600 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T20:51:14.601 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T20:51:14.661 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3590.g7b710596.el9 ceph 6.6 k 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.4 M 2024-05-10T20:51:14.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 863 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3590.g7b710596.el9 ceph 147 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3590.g7b710596.el9 ceph 1.5 M 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 142 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 3.5 M 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 7.4 M 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 49 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 10 M 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3590.g7b710596.el9 ceph 43 M 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 273 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 764 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 224 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-3590.g7b710596.el9 ceph 27 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 18 k 2024-05-10T20:51:14.673 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 32 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 756 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 127 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3590.g7b710596.el9 ceph 162 k 2024-05-10T20:51:14.674 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3590.g7b710596.el9 ceph 321 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 300 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 99 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 86 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.0 M 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 169 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.3 M 2024-05-10T20:51:14.675 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.1 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 20 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 34 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3590.g7b710596.el9 ceph 2.3 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 274 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3590.g7b710596.el9 ceph 4.9 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 18 M 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 16 k 2024-05-10T20:51:14.676 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3590.g7b710596.el9 ceph 25 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3590.g7b710596.el9 ceph 157 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T20:51:14.677 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 497 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.2 M 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T20:51:14.678 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 45 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 135 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T20:51:14.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T20:51:14.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T20:51:14.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T20:51:14.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T20:51:14.683 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-05-10T20:51:14.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:51:14.685 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 219 M 2024-05-10T20:51:14.685 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-10T20:51:14.741 INFO:teuthology.orchestra.run.smithi142.stdout:(5/156): ceph-mgr-19.0.0-3590.g7b710596.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-05-10T20:51:15.150 INFO:teuthology.orchestra.run.smithi142.stdout:(6/156): ceph-mds-19.0.0-3590.g7b710596.el9.x86 3.3 MB/s | 2.3 MB 00:00 2024-05-10T20:51:15.225 INFO:teuthology.orchestra.run.smithi142.stdout:(7/156): ceph-mon-19.0.0-3590.g7b710596.el9.x86 10 MB/s | 4.9 MB 00:00 2024-05-10T20:51:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:(1/156): ceph-19.0.0-3590.g7b710596.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-10T20:51:15.801 INFO:teuthology.orchestra.run.smithi142.stdout:(8/156): ceph-common-19.0.0-3590.g7b710596.el9. 9.8 MB/s | 20 MB 00:02 2024-05-10T20:51:15.851 INFO:teuthology.orchestra.run.smithi142.stdout:(9/156): ceph-selinux-19.0.0-3590.g7b710596.el9 504 kB/s | 25 kB 00:00 2024-05-10T20:51:15.954 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-05-10T20:51:16.110 INFO:teuthology.orchestra.run.smithi142.stdout:(10/156): ceph-radosgw-19.0.0-3590.g7b710596.el 11 MB/s | 10 MB 00:00 2024-05-10T20:51:16.241 INFO:teuthology.orchestra.run.smithi092.stdout:(2/156): ceph-fuse-19.0.0-3590.g7b710596.el9.x8 1.0 MB/s | 863 kB 00:00 2024-05-10T20:51:16.252 INFO:teuthology.orchestra.run.smithi142.stdout:(11/156): cephfs-mirror-19.0.0-3590.g7b710596.e 1.5 MB/s | 224 kB 00:00 2024-05-10T20:51:16.335 INFO:teuthology.orchestra.run.smithi142.stdout:(12/156): cephfs-shell-19.0.0-3590.g7b710596.el 325 kB/s | 27 kB 00:00 2024-05-10T20:51:16.375 INFO:teuthology.orchestra.run.smithi092.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.1 MB/s | 147 kB 00:00 2024-05-10T20:51:16.411 INFO:teuthology.orchestra.run.smithi142.stdout:(13/156): libcephfs-devel-19.0.0-3590.g7b710596 423 kB/s | 32 kB 00:00 2024-05-10T20:51:16.702 INFO:teuthology.orchestra.run.smithi142.stdout:(14/156): libcephfs2-19.0.0-3590.g7b710596.el9. 2.5 MB/s | 756 kB 00:00 2024-05-10T20:51:16.794 INFO:teuthology.orchestra.run.smithi142.stdout:(15/156): libcephsqlite-19.0.0-3590.g7b710596.e 1.7 MB/s | 157 kB 00:00 2024-05-10T20:51:16.895 INFO:teuthology.orchestra.run.smithi142.stdout:(16/156): librados-devel-19.0.0-3590.g7b710596. 1.2 MB/s | 127 kB 00:00 2024-05-10T20:51:17.078 INFO:teuthology.orchestra.run.smithi142.stdout:(17/156): libradosstriper1-19.0.0-3590.g7b71059 2.6 MB/s | 497 kB 00:00 2024-05-10T20:51:17.820 INFO:teuthology.orchestra.run.smithi142.stdout:(18/156): ceph-osd-19.0.0-3590.g7b710596.el9.x8 6.6 MB/s | 18 MB 00:02 2024-05-10T20:51:17.912 INFO:teuthology.orchestra.run.smithi142.stdout:(19/156): python3-ceph-argparse-19.0.0-3590.g7b 494 kB/s | 45 kB 00:00 2024-05-10T20:51:18.017 INFO:teuthology.orchestra.run.smithi092.stdout:(4/156): ceph-mds-19.0.0-3590.g7b710596.el9.x86 1.4 MB/s | 2.3 MB 00:01 2024-05-10T20:51:18.104 INFO:teuthology.orchestra.run.smithi142.stdout:(20/156): librgw2-19.0.0-3590.g7b710596.el9.x86 5.0 MB/s | 5.2 MB 00:01 2024-05-10T20:51:18.129 INFO:teuthology.orchestra.run.smithi142.stdout:(21/156): python3-ceph-common-19.0.0-3590.g7b71 625 kB/s | 135 kB 00:00 2024-05-10T20:51:18.179 INFO:teuthology.orchestra.run.smithi142.stdout:(22/156): python3-cephfs-19.0.0-3590.g7b710596. 2.1 MB/s | 162 kB 00:00 2024-05-10T20:51:18.263 INFO:teuthology.orchestra.run.smithi142.stdout:(23/156): python3-rbd-19.0.0-3590.g7b710596.el9 3.5 MB/s | 300 kB 00:00 2024-05-10T20:51:18.288 INFO:teuthology.orchestra.run.smithi142.stdout:(24/156): python3-rados-19.0.0-3590.g7b710596.e 2.0 MB/s | 321 kB 00:00 2024-05-10T20:51:18.355 INFO:teuthology.orchestra.run.smithi142.stdout:(25/156): python3-rgw-19.0.0-3590.g7b710596.el9 1.1 MB/s | 99 kB 00:00 2024-05-10T20:51:18.380 INFO:teuthology.orchestra.run.smithi142.stdout:(26/156): rbd-fuse-19.0.0-3590.g7b710596.el9.x8 938 kB/s | 86 kB 00:00 2024-05-10T20:51:18.408 INFO:teuthology.orchestra.run.smithi092.stdout:(5/156): ceph-base-19.0.0-3590.g7b710596.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-10T20:51:18.489 INFO:teuthology.orchestra.run.smithi142.stdout:(27/156): rbd-nbd-19.0.0-3590.g7b710596.el9.x86 1.5 MB/s | 169 kB 00:00 2024-05-10T20:51:18.589 INFO:teuthology.orchestra.run.smithi142.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3590.g 342 kB/s | 34 kB 00:00 2024-05-10T20:51:18.692 INFO:teuthology.orchestra.run.smithi092.stdout:(6/156): ceph-mgr-19.0.0-3590.g7b710596.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-10T20:51:18.723 INFO:teuthology.orchestra.run.smithi142.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3590.g7b71059 1.0 MB/s | 142 kB 00:00 2024-05-10T20:51:18.848 INFO:teuthology.orchestra.run.smithi142.stdout:(30/156): rbd-mirror-19.0.0-3590.g7b710596.el9. 6.2 MB/s | 3.0 MB 00:00 2024-05-10T20:51:19.573 INFO:teuthology.orchestra.run.smithi142.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3590.g7b710 4.2 MB/s | 3.5 MB 00:00 2024-05-10T20:51:19.682 INFO:teuthology.orchestra.run.smithi142.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3590.g7b 2.5 MB/s | 274 kB 00:00 2024-05-10T20:51:19.832 INFO:teuthology.orchestra.run.smithi142.stdout:(33/156): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-05-10T20:51:19.849 INFO:teuthology.orchestra.run.smithi142.stdout:(34/156): ceph-mgr-rook-19.0.0-3590.g7b710596.e 295 kB/s | 49 kB 00:00 2024-05-10T20:51:19.875 INFO:teuthology.orchestra.run.smithi092.stdout:(7/156): ceph-mon-19.0.0-3590.g7b710596.el9.x86 3.3 MB/s | 4.9 MB 00:01 2024-05-10T20:51:20.216 INFO:teuthology.orchestra.run.smithi142.stdout:(35/156): ceph-test-19.0.0-3590.g7b710596.el9.x 9.9 MB/s | 43 MB 00:04 2024-05-10T20:51:20.233 INFO:teuthology.orchestra.run.smithi142.stdout:(36/156): ceph-prometheus-alerts-19.0.0-3590.g7 41 kB/s | 16 kB 00:00 2024-05-10T20:51:20.258 INFO:teuthology.orchestra.run.smithi142.stdout:(37/156): ceph-volume-19.0.0-3590.g7b710596.el9 668 kB/s | 273 kB 00:00 2024-05-10T20:51:20.275 INFO:teuthology.orchestra.run.smithi142.stdout:(38/156): cephfs-top-19.0.0-3590.g7b710596.el9. 435 kB/s | 18 kB 00:00 2024-05-10T20:51:20.309 INFO:teuthology.orchestra.run.smithi142.stdout:(39/156): cephadm-19.0.0-3590.g7b710596.el9.noa 8.1 MB/s | 764 kB 00:00 2024-05-10T20:51:20.334 INFO:teuthology.orchestra.run.smithi092.stdout:(8/156): ceph-common-19.0.0-3590.g7b710596.el9. 3.7 MB/s | 20 MB 00:05 2024-05-10T20:51:20.367 INFO:teuthology.orchestra.run.smithi092.stdout:(9/156): ceph-selinux-19.0.0-3590.g7b710596.el9 761 kB/s | 25 kB 00:00 2024-05-10T20:51:20.401 INFO:teuthology.orchestra.run.smithi142.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-05-10T20:51:20.451 INFO:teuthology.orchestra.run.smithi142.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-10T20:51:20.493 INFO:teuthology.orchestra.run.smithi142.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-10T20:51:20.535 INFO:teuthology.orchestra.run.smithi142.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-10T20:51:20.585 INFO:teuthology.orchestra.run.smithi142.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-05-10T20:51:20.619 INFO:teuthology.orchestra.run.smithi142.stdout:(45/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-10T20:51:20.644 INFO:teuthology.orchestra.run.smithi142.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-10T20:51:20.669 INFO:teuthology.orchestra.run.smithi142.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-10T20:51:20.695 INFO:teuthology.orchestra.run.smithi142.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-10T20:51:20.762 INFO:teuthology.orchestra.run.smithi142.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-10T20:51:20.795 INFO:teuthology.orchestra.run.smithi142.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-05-10T20:51:20.820 INFO:teuthology.orchestra.run.smithi142.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-05-10T20:51:20.862 INFO:teuthology.orchestra.run.smithi142.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-10T20:51:20.888 INFO:teuthology.orchestra.run.smithi142.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-05-10T20:51:20.921 INFO:teuthology.orchestra.run.smithi142.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-10T20:51:20.938 INFO:teuthology.orchestra.run.smithi142.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-10T20:51:20.988 INFO:teuthology.orchestra.run.smithi142.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-10T20:51:21.014 INFO:teuthology.orchestra.run.smithi142.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-10T20:51:21.064 INFO:teuthology.orchestra.run.smithi142.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-05-10T20:51:21.089 INFO:teuthology.orchestra.run.smithi142.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-05-10T20:51:21.139 INFO:teuthology.orchestra.run.smithi142.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-10T20:51:21.165 INFO:teuthology.orchestra.run.smithi142.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-10T20:51:21.198 INFO:teuthology.orchestra.run.smithi142.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-05-10T20:51:21.223 INFO:teuthology.orchestra.run.smithi142.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-05-10T20:51:21.249 INFO:teuthology.orchestra.run.smithi142.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-10T20:51:21.291 INFO:teuthology.orchestra.run.smithi142.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-05-10T20:51:21.508 INFO:teuthology.orchestra.run.smithi142.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 610 kB/s | 188 kB 00:00 2024-05-10T20:51:21.699 INFO:teuthology.orchestra.run.smithi142.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-10T20:51:21.818 INFO:teuthology.orchestra.run.smithi092.stdout:(10/156): ceph-radosgw-19.0.0-3590.g7b710596.el 5.2 MB/s | 10 MB 00:01 2024-05-10T20:51:21.850 INFO:teuthology.orchestra.run.smithi142.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 7.9 MB/s | 4.7 MB 00:00 2024-05-10T20:51:21.884 INFO:teuthology.orchestra.run.smithi142.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-10T20:51:21.935 INFO:teuthology.orchestra.run.smithi092.stdout:(11/156): cephfs-mirror-19.0.0-3590.g7b710596.e 1.9 MB/s | 224 kB 00:00 2024-05-10T20:51:21.950 INFO:teuthology.orchestra.run.smithi142.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-05-10T20:51:21.969 INFO:teuthology.orchestra.run.smithi142.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-10T20:51:22.001 INFO:teuthology.orchestra.run.smithi092.stdout:(12/156): cephfs-shell-19.0.0-3590.g7b710596.el 407 kB/s | 27 kB 00:00 2024-05-10T20:51:22.060 INFO:teuthology.orchestra.run.smithi092.stdout:(13/156): libcephfs-devel-19.0.0-3590.g7b710596 545 kB/s | 32 kB 00:00 2024-05-10T20:51:22.093 INFO:teuthology.orchestra.run.smithi142.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-10T20:51:22.118 INFO:teuthology.orchestra.run.smithi142.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 582 kB/s | 87 kB 00:00 2024-05-10T20:51:22.202 INFO:teuthology.orchestra.run.smithi092.stdout:(14/156): libcephfs2-19.0.0-3590.g7b710596.el9. 5.2 MB/s | 756 kB 00:00 2024-05-10T20:51:22.218 INFO:teuthology.orchestra.run.smithi142.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-10T20:51:22.269 INFO:teuthology.orchestra.run.smithi092.stdout:(15/156): libcephsqlite-19.0.0-3590.g7b710596.e 2.3 MB/s | 157 kB 00:00 2024-05-10T20:51:22.335 INFO:teuthology.orchestra.run.smithi142.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-05-10T20:51:22.336 INFO:teuthology.orchestra.run.smithi092.stdout:(16/156): librados-devel-19.0.0-3590.g7b710596. 1.9 MB/s | 127 kB 00:00 2024-05-10T20:51:22.361 INFO:teuthology.orchestra.run.smithi142.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-05-10T20:51:22.411 INFO:teuthology.orchestra.run.smithi142.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-05-10T20:51:22.436 INFO:teuthology.orchestra.run.smithi142.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-05-10T20:51:22.486 INFO:teuthology.orchestra.run.smithi092.stdout:(17/156): libradosstriper1-19.0.0-3590.g7b71059 3.2 MB/s | 497 kB 00:00 2024-05-10T20:51:22.503 INFO:teuthology.orchestra.run.smithi142.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-10T20:51:22.528 INFO:teuthology.orchestra.run.smithi142.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 379 kB/s | 35 kB 00:00 2024-05-10T20:51:22.553 INFO:teuthology.orchestra.run.smithi142.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-05-10T20:51:22.604 INFO:teuthology.orchestra.run.smithi142.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-10T20:51:22.671 INFO:teuthology.orchestra.run.smithi142.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-05-10T20:51:22.696 INFO:teuthology.orchestra.run.smithi142.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 842 kB/s | 77 kB 00:00 2024-05-10T20:51:22.721 INFO:teuthology.orchestra.run.smithi142.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-10T20:51:22.746 INFO:teuthology.orchestra.run.smithi142.stdout:(86/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-05-10T20:51:22.772 INFO:teuthology.orchestra.run.smithi142.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-10T20:51:22.797 INFO:teuthology.orchestra.run.smithi142.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-10T20:51:22.822 INFO:teuthology.orchestra.run.smithi142.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-10T20:51:22.839 INFO:teuthology.orchestra.run.smithi142.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-05-10T20:51:22.864 INFO:teuthology.orchestra.run.smithi142.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-10T20:51:22.890 INFO:teuthology.orchestra.run.smithi142.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-05-10T20:51:22.949 INFO:teuthology.orchestra.run.smithi142.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-10T20:51:22.973 INFO:teuthology.orchestra.run.smithi142.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 406 kB/s | 44 kB 00:00 2024-05-10T20:51:22.995 INFO:teuthology.orchestra.run.smithi092.stdout:(18/156): ceph-osd-19.0.0-3590.g7b710596.el9.x8 4.1 MB/s | 18 MB 00:04 2024-05-10T20:51:22.999 INFO:teuthology.orchestra.run.smithi142.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-10T20:51:23.041 INFO:teuthology.orchestra.run.smithi142.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-10T20:51:23.062 INFO:teuthology.orchestra.run.smithi092.stdout:(19/156): python3-ceph-argparse-19.0.0-3590.g7b 680 kB/s | 45 kB 00:00 2024-05-10T20:51:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-10T20:51:23.099 INFO:teuthology.orchestra.run.smithi142.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-05-10T20:51:23.129 INFO:teuthology.orchestra.run.smithi092.stdout:(20/156): python3-ceph-common-19.0.0-3590.g7b71 2.0 MB/s | 135 kB 00:00 2024-05-10T20:51:23.150 INFO:teuthology.orchestra.run.smithi142.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-10T20:51:23.175 INFO:teuthology.orchestra.run.smithi142.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 860 kB/s | 94 kB 00:00 2024-05-10T20:51:23.204 INFO:teuthology.orchestra.run.smithi092.stdout:(21/156): librgw2-19.0.0-3590.g7b710596.el9.x86 7.2 MB/s | 5.2 MB 00:00 2024-05-10T20:51:23.229 INFO:teuthology.orchestra.run.smithi092.stdout:(22/156): python3-cephfs-19.0.0-3590.g7b710596. 1.6 MB/s | 162 kB 00:00 2024-05-10T20:51:23.242 INFO:teuthology.orchestra.run.smithi142.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 487 kB/s | 45 kB 00:00 2024-05-10T20:51:23.267 INFO:teuthology.orchestra.run.smithi142.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-10T20:51:23.271 INFO:teuthology.orchestra.run.smithi092.stdout:(23/156): python3-rados-19.0.0-3590.g7b710596.e 4.7 MB/s | 321 kB 00:00 2024-05-10T20:51:23.338 INFO:teuthology.orchestra.run.smithi092.stdout:(24/156): python3-rbd-19.0.0-3590.g7b710596.el9 2.7 MB/s | 300 kB 00:00 2024-05-10T20:51:23.363 INFO:teuthology.orchestra.run.smithi092.stdout:(25/156): python3-rgw-19.0.0-3590.g7b710596.el9 1.1 MB/s | 99 kB 00:00 2024-05-10T20:51:23.418 INFO:teuthology.orchestra.run.smithi142.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-10T20:51:23.422 INFO:teuthology.orchestra.run.smithi092.stdout:(26/156): rbd-fuse-19.0.0-3590.g7b710596.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-10T20:51:23.522 INFO:teuthology.orchestra.run.smithi092.stdout:(27/156): rbd-nbd-19.0.0-3590.g7b710596.el9.x86 1.7 MB/s | 169 kB 00:00 2024-05-10T20:51:23.551 INFO:teuthology.orchestra.run.smithi142.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-05-10T20:51:23.589 INFO:teuthology.orchestra.run.smithi092.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3590.g 513 kB/s | 34 kB 00:00 2024-05-10T20:51:23.664 INFO:teuthology.orchestra.run.smithi092.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3590.g7b71059 1.9 MB/s | 142 kB 00:00 2024-05-10T20:51:23.773 INFO:teuthology.orchestra.run.smithi092.stdout:(30/156): rbd-mirror-19.0.0-3590.g7b710596.el9. 7.4 MB/s | 3.0 MB 00:00 2024-05-10T20:51:23.802 INFO:teuthology.orchestra.run.smithi142.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-05-10T20:51:23.835 INFO:teuthology.orchestra.run.smithi142.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 966 kB/s | 548 kB 00:00 2024-05-10T20:51:23.902 INFO:teuthology.orchestra.run.smithi142.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 677 kB/s | 309 kB 00:00 2024-05-10T20:51:23.927 INFO:teuthology.orchestra.run.smithi142.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-10T20:51:23.986 INFO:teuthology.orchestra.run.smithi142.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-05-10T20:51:24.003 INFO:teuthology.orchestra.run.smithi142.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-05-10T20:51:24.078 INFO:teuthology.orchestra.run.smithi142.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-10T20:51:24.170 INFO:teuthology.orchestra.run.smithi142.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-10T20:51:24.245 INFO:teuthology.orchestra.run.smithi142.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 581 kB/s | 43 kB 00:00 2024-05-10T20:51:24.271 INFO:teuthology.orchestra.run.smithi142.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-05-10T20:51:24.282 INFO:teuthology.orchestra.run.smithi092.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3590.g7b710 5.7 MB/s | 3.5 MB 00:00 2024-05-10T20:51:24.338 INFO:teuthology.orchestra.run.smithi142.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-05-10T20:51:24.435 INFO:teuthology.orchestra.run.smithi092.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3590.g7b 4.0 MB/s | 274 kB 00:00 2024-05-10T20:51:24.435 INFO:teuthology.orchestra.run.smithi092.stdout:(33/156): ceph-mgr-rook-19.0.0-3590.g7b710596.e 659 kB/s | 49 kB 00:00 2024-05-10T20:51:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-10T20:51:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2024-05-10T20:51:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-10T20:51:24.455 INFO:teuthology.orchestra.run.smithi142.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-10T20:51:24.483 INFO:teuthology.orchestra.run.smithi092.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3590.g7 282 kB/s | 16 kB 00:00 2024-05-10T20:51:24.497 INFO:teuthology.orchestra.run.smithi142.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-10T20:51:24.522 INFO:teuthology.orchestra.run.smithi142.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-05-10T20:51:24.548 INFO:teuthology.orchestra.run.smithi142.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-10T20:51:24.615 INFO:teuthology.orchestra.run.smithi142.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-10T20:51:24.640 INFO:teuthology.orchestra.run.smithi142.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-10T20:51:24.690 INFO:teuthology.orchestra.run.smithi142.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-10T20:51:24.716 INFO:teuthology.orchestra.run.smithi142.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-10T20:51:24.740 INFO:teuthology.orchestra.run.smithi142.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-10T20:51:24.766 INFO:teuthology.orchestra.run.smithi142.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-10T20:51:24.816 INFO:teuthology.orchestra.run.smithi142.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-05-10T20:51:24.841 INFO:teuthology.orchestra.run.smithi142.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-05-10T20:51:24.883 INFO:teuthology.orchestra.run.smithi142.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-10T20:51:24.908 INFO:teuthology.orchestra.run.smithi142.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 857 kB/s | 79 kB 00:00 2024-05-10T20:51:24.934 INFO:teuthology.orchestra.run.smithi142.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-05-10T20:51:24.949 INFO:teuthology.orchestra.run.smithi092.stdout:(35/156): ceph-test-19.0.0-3590.g7b710596.el9.x 9.4 MB/s | 43 MB 00:04 2024-05-10T20:51:24.961 INFO:teuthology.orchestra.run.smithi142.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-10T20:51:24.975 INFO:teuthology.orchestra.run.smithi092.stdout:(36/156): ceph-volume-19.0.0-3590.g7b710596.el9 555 kB/s | 273 kB 00:00 2024-05-10T20:51:24.984 INFO:teuthology.orchestra.run.smithi142.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-10T20:51:25.001 INFO:teuthology.orchestra.run.smithi142.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-10T20:51:25.043 INFO:teuthology.orchestra.run.smithi142.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-05-10T20:51:25.060 INFO:teuthology.orchestra.run.smithi142.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-10T20:51:25.075 INFO:teuthology.orchestra.run.smithi092.stdout:(37/156): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-05-10T20:51:25.086 INFO:teuthology.orchestra.run.smithi142.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-10T20:51:25.101 INFO:teuthology.orchestra.run.smithi092.stdout:(38/156): cephfs-top-19.0.0-3590.g7b710596.el9. 145 kB/s | 18 kB 00:00 2024-05-10T20:51:25.111 INFO:teuthology.orchestra.run.smithi142.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 879 kB/s | 59 kB 00:00 2024-05-10T20:51:25.134 INFO:teuthology.orchestra.run.smithi092.stdout:(39/156): cephadm-19.0.0-3590.g7b710596.el9.noa 4.0 MB/s | 764 kB 00:00 2024-05-10T20:51:25.136 INFO:teuthology.orchestra.run.smithi142.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-10T20:51:25.161 INFO:teuthology.orchestra.run.smithi142.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-10T20:51:25.187 INFO:teuthology.orchestra.run.smithi142.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-10T20:51:25.220 INFO:teuthology.orchestra.run.smithi142.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-10T20:51:25.245 INFO:teuthology.orchestra.run.smithi142.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-10T20:51:25.271 INFO:teuthology.orchestra.run.smithi142.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-10T20:51:25.321 INFO:teuthology.orchestra.run.smithi142.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-10T20:51:25.340 INFO:teuthology.orchestra.run.smithi092.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 304 kB/s | 80 kB 00:00 2024-05-10T20:51:25.363 INFO:teuthology.orchestra.run.smithi142.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-10T20:51:25.370 INFO:teuthology.orchestra.run.smithi092.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 836 kB/s | 195 kB 00:00 2024-05-10T20:51:25.410 INFO:teuthology.orchestra.run.smithi092.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-05-10T20:51:25.469 INFO:teuthology.orchestra.run.smithi092.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-10T20:51:25.494 INFO:teuthology.orchestra.run.smithi092.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-05-10T20:51:25.544 INFO:teuthology.orchestra.run.smithi092.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-05-10T20:51:25.570 INFO:teuthology.orchestra.run.smithi092.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-10T20:51:25.595 INFO:teuthology.orchestra.run.smithi092.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-10T20:51:25.620 INFO:teuthology.orchestra.run.smithi092.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-10T20:51:25.621 INFO:teuthology.orchestra.run.smithi142.stdout:(149/156): librbd1-19.0.0-3590.g7b710596.el9.x8 10 MB/s | 3.1 MB 00:00 2024-05-10T20:51:25.663 INFO:teuthology.orchestra.run.smithi142.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 7.7 MB/s | 2.3 MB 00:00 2024-05-10T20:51:25.713 INFO:teuthology.orchestra.run.smithi142.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 838 kB/s | 42 kB 00:00 2024-05-10T20:51:25.797 INFO:teuthology.orchestra.run.smithi142.stdout:(152/156): librados2-19.0.0-3590.g7b710596.el9. 6.3 MB/s | 3.3 MB 00:00 2024-05-10T20:51:25.814 INFO:teuthology.orchestra.run.smithi142.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-05-10T20:51:25.881 INFO:teuthology.orchestra.run.smithi142.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-05-10T20:51:25.948 INFO:teuthology.orchestra.run.smithi142.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-05-10T20:51:26.040 INFO:teuthology.orchestra.run.smithi142.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-05-10T20:51:26.045 INFO:teuthology.orchestra.run.smithi092.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-05-10T20:51:26.046 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-05-10T20:51:26.046 INFO:teuthology.orchestra.run.smithi142.stdout:Total 17 MB/s | 219 MB 00:12 2024-05-10T20:51:26.162 INFO:teuthology.orchestra.run.smithi092.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-05-10T20:51:26.437 INFO:teuthology.orchestra.run.smithi092.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-10T20:51:26.462 INFO:teuthology.orchestra.run.smithi092.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-05-10T20:51:26.488 INFO:teuthology.orchestra.run.smithi092.stdout:(53/156): automake-1.16.2-8.el9.noarch.rpm 750 kB/s | 689 kB 00:00 2024-05-10T20:51:26.538 INFO:teuthology.orchestra.run.smithi092.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 325 kB/s | 33 kB 00:00 2024-05-10T20:51:26.597 INFO:teuthology.orchestra.run.smithi092.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-10T20:51:26.639 INFO:teuthology.orchestra.run.smithi092.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-10T20:51:26.705 INFO:teuthology.orchestra.run.smithi092.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-10T20:51:26.748 INFO:teuthology.orchestra.run.smithi092.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-05-10T20:51:26.931 INFO:teuthology.orchestra.run.smithi092.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-05-10T20:51:26.981 INFO:teuthology.orchestra.run.smithi092.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-05-10T20:51:27.032 INFO:teuthology.orchestra.run.smithi092.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.3 MB/s | 3.0 MB 00:00 2024-05-10T20:51:27.057 INFO:teuthology.orchestra.run.smithi092.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 202 kB/s | 25 kB 00:00 2024-05-10T20:51:27.091 INFO:teuthology.orchestra.run.smithi092.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-05-10T20:51:27.149 INFO:teuthology.orchestra.run.smithi092.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-10T20:51:27.174 INFO:teuthology.orchestra.run.smithi092.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-10T20:51:27.209 INFO:teuthology.orchestra.run.smithi092.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 300 kB/s | 35 kB 00:00 2024-05-10T20:51:27.229 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-05-10T20:51:27.379 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-05-10T20:51:27.379 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-05-10T20:51:27.583 INFO:teuthology.orchestra.run.smithi092.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-10T20:51:27.717 INFO:teuthology.orchestra.run.smithi092.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-10T20:51:27.834 INFO:teuthology.orchestra.run.smithi092.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-10T20:51:29.123 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-05-10T20:51:29.123 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-05-10T20:51:29.601 INFO:teuthology.orchestra.run.smithi092.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-05-10T20:51:29.801 INFO:teuthology.orchestra.run.smithi092.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-05-10T20:51:29.959 INFO:teuthology.orchestra.run.smithi092.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 552 kB/s | 87 kB 00:00 2024-05-10T20:51:31.152 INFO:teuthology.orchestra.run.smithi092.stdout:(73/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.2 MB/s | 4.7 MB 00:04 2024-05-10T20:51:31.493 INFO:teuthology.orchestra.run.smithi092.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 715 kB/s | 244 kB 00:00 2024-05-10T20:51:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 708 kB/s | 248 kB 00:00 2024-05-10T20:51:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-10T20:51:32.044 INFO:teuthology.orchestra.run.smithi092.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-10T20:51:32.146 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:15 2024-05-10T20:51:32.328 INFO:teuthology.orchestra.run.smithi092.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 608 kB/s | 172 kB 00:00 2024-05-10T20:51:32.428 INFO:teuthology.orchestra.run.smithi092.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-05-10T20:51:33.812 INFO:teuthology.orchestra.run.smithi092.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 990 kB/s | 5.8 MB 00:05 2024-05-10T20:51:34.003 INFO:teuthology.orchestra.run.smithi092.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-05-10T20:51:34.112 INFO:teuthology.orchestra.run.smithi092.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-10T20:51:34.237 INFO:teuthology.orchestra.run.smithi092.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 618 kB/s | 77 kB 00:00 2024-05-10T20:51:34.380 INFO:teuthology.orchestra.run.smithi092.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 298 kB/s | 42 kB 00:00 2024-05-10T20:51:34.521 INFO:teuthology.orchestra.run.smithi092.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-05-10T20:51:35.377 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-05-10T20:51:35.453 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T20:51:35.491 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T20:51:35.544 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T20:51:35.578 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T20:51:35.604 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T20:51:35.604 INFO:teuthology.orchestra.run.smithi092.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 147 kB/s | 159 kB 00:01 2024-05-10T20:51:35.646 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T20:51:35.722 INFO:teuthology.orchestra.run.smithi092.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-10T20:51:35.855 INFO:teuthology.orchestra.run.smithi092.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 383 kB/s | 51 kB 00:00 2024-05-10T20:51:35.955 INFO:teuthology.orchestra.run.smithi092.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.0 MB/s | 6.0 MB 00:05 2024-05-10T20:51:35.972 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T20:51:35.981 INFO:teuthology.orchestra.run.smithi092.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-05-10T20:51:36.079 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:51:36.089 INFO:teuthology.orchestra.run.smithi092.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-05-10T20:51:36.204 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:51:36.206 INFO:teuthology.orchestra.run.smithi092.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 379 kB/s | 44 kB 00:00 2024-05-10T20:51:36.230 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:51:36.315 INFO:teuthology.orchestra.run.smithi092.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-10T20:51:36.322 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:51:36.351 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:51:36.432 INFO:teuthology.orchestra.run.smithi092.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-05-10T20:51:36.500 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T20:51:36.541 INFO:teuthology.orchestra.run.smithi092.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-10T20:51:36.556 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T20:51:36.584 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:51:36.681 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:51:36.683 INFO:teuthology.orchestra.run.smithi092.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-05-10T20:51:36.710 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:51:36.756 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:51:36.778 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:51:36.803 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:51:36.816 INFO:teuthology.orchestra.run.smithi092.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 434 kB/s | 58 kB 00:00 2024-05-10T20:51:36.925 INFO:teuthology.orchestra.run.smithi092.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 864 kB/s | 94 kB 00:00 2024-05-10T20:51:36.979 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:51:37.034 INFO:teuthology.orchestra.run.smithi092.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-05-10T20:51:37.105 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:51:37.142 INFO:teuthology.orchestra.run.smithi092.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-05-10T20:51:37.226 INFO:teuthology.orchestra.run.smithi092.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-10T20:51:37.267 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:51:37.327 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:51:37.359 INFO:teuthology.orchestra.run.smithi092.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-05-10T20:51:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:51:37.512 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:51:37.711 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:51:37.726 INFO:teuthology.orchestra.run.smithi092.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-10T20:51:37.822 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T20:51:37.849 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T20:51:37.851 INFO:teuthology.orchestra.run.smithi092.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-10T20:51:37.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-10T20:51:37.952 INFO:teuthology.orchestra.run.smithi092.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-10T20:51:37.958 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-10T20:51:38.000 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-10T20:51:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-10T20:51:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-10T20:51:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-10T20:51:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-10T20:51:38.219 INFO:teuthology.orchestra.run.smithi092.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-10T20:51:38.234 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-10T20:51:38.265 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-10T20:51:38.300 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-10T20:51:38.310 INFO:teuthology.orchestra.run.smithi092.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-05-10T20:51:38.376 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-10T20:51:38.394 INFO:teuthology.orchestra.run.smithi092.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-05-10T20:51:38.411 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 32/164 2024-05-10T20:51:38.454 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 33/164 2024-05-10T20:51:38.483 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 34/164 2024-05-10T20:51:38.486 INFO:teuthology.orchestra.run.smithi092.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-05-10T20:51:38.554 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-10T20:51:38.561 INFO:teuthology.orchestra.run.smithi092.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-10T20:51:38.584 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-10T20:51:38.639 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-10T20:51:38.654 INFO:teuthology.orchestra.run.smithi092.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-10T20:51:38.713 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-10T20:51:38.770 INFO:teuthology.orchestra.run.smithi092.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-05-10T20:51:38.837 INFO:teuthology.orchestra.run.smithi092.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-05-10T20:51:38.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-10T20:51:38.900 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-10T20:51:38.913 INFO:teuthology.orchestra.run.smithi092.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-05-10T20:51:38.988 INFO:teuthology.orchestra.run.smithi092.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-05-10T20:51:39.063 INFO:teuthology.orchestra.run.smithi092.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-10T20:51:39.138 INFO:teuthology.orchestra.run.smithi092.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-10T20:51:39.239 INFO:teuthology.orchestra.run.smithi092.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-10T20:51:39.331 INFO:teuthology.orchestra.run.smithi092.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-10T20:51:39.421 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-10T20:51:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-10T20:51:39.505 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-10T20:51:39.533 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-10T20:51:39.565 INFO:teuthology.orchestra.run.smithi092.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-10T20:51:39.576 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-10T20:51:39.614 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 44/164 2024-05-10T20:51:39.640 INFO:teuthology.orchestra.run.smithi092.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-10T20:51:39.655 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-10T20:51:39.743 INFO:teuthology.orchestra.run.smithi092.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 104 kB/s | 11 kB 00:00 2024-05-10T20:51:39.816 INFO:teuthology.orchestra.run.smithi092.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 243 kB/s | 18 kB 00:00 2024-05-10T20:51:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-10T20:51:39.875 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-10T20:51:39.883 INFO:teuthology.orchestra.run.smithi092.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 348 kB/s | 23 kB 00:00 2024-05-10T20:51:39.950 INFO:teuthology.orchestra.run.smithi092.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 292 kB/s | 19 kB 00:00 2024-05-10T20:51:39.965 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-10T20:51:40.023 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-10T20:51:40.050 INFO:teuthology.orchestra.run.smithi092.stdout:(127/156): python3-numpy-1.20.1-5.el9.x86_64.rp 685 kB/s | 5.1 MB 00:07 2024-05-10T20:51:40.067 INFO:teuthology.orchestra.run.smithi092.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-05-10T20:51:40.095 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T20:51:40.126 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T20:51:40.134 INFO:teuthology.orchestra.run.smithi092.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-05-10T20:51:40.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T20:51:40.210 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T20:51:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 948 kB/s | 79 kB 00:00 2024-05-10T20:51:40.246 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T20:51:40.288 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T20:51:40.301 INFO:teuthology.orchestra.run.smithi092.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-10T20:51:40.330 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T20:51:40.385 INFO:teuthology.orchestra.run.smithi092.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-10T20:51:40.468 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T20:51:40.577 INFO:teuthology.orchestra.run.smithi092.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 86 kB/s | 16 kB 00:00 2024-05-10T20:51:40.642 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T20:51:40.660 INFO:teuthology.orchestra.run.smithi092.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-10T20:51:40.753 INFO:teuthology.orchestra.run.smithi092.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-10T20:51:40.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T20:51:40.794 INFO:teuthology.orchestra.run.smithi092.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-05-10T20:51:40.836 INFO:teuthology.orchestra.run.smithi092.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-05-10T20:51:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T20:51:40.920 INFO:teuthology.orchestra.run.smithi092.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-05-10T20:51:40.945 INFO:teuthology.orchestra.run.smithi092.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-05-10T20:51:41.002 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T20:51:41.012 INFO:teuthology.orchestra.run.smithi092.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-05-10T20:51:41.041 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T20:51:41.062 INFO:teuthology.orchestra.run.smithi092.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-05-10T20:51:41.081 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T20:51:41.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T20:51:41.129 INFO:teuthology.orchestra.run.smithi092.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-10T20:51:41.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T20:51:41.165 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:41.166 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T20:51:41.180 INFO:teuthology.orchestra.run.smithi092.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-05-10T20:51:41.187 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T20:51:41.188 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T20:51:41.190 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T20:51:41.191 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T20:51:41.193 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T20:51:41.197 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T20:51:41.198 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T20:51:41.204 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T20:51:41.213 INFO:teuthology.orchestra.run.smithi092.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-10T20:51:41.255 INFO:teuthology.orchestra.run.smithi092.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-05-10T20:51:41.273 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-3590.g7b710596.el9 ceph 6.6 k 2024-05-10T20:51:41.294 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.4 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 863 k 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3590.g7b710596.el9 ceph 147 k 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-3590.g7b710596.el9 ceph 1.5 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 142 k 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 3.5 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 7.4 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 49 k 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 10 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-3590.g7b710596.el9 ceph 43 M 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 273 k 2024-05-10T20:51:41.295 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 764 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 224 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-3590.g7b710596.el9 ceph 27 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 18 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 32 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 756 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-3590.g7b710596.el9 ceph 127 k 2024-05-10T20:51:41.296 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-3590.g7b710596.el9 ceph 162 k 2024-05-10T20:51:41.297 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-3590.g7b710596.el9 ceph 321 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 300 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-3590.g7b710596.el9 ceph 99 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 86 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.0 M 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 169 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T20:51:41.298 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.3 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 3.1 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T20:51:41.299 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 20 M 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 34 k 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-3590.g7b710596.el9 ceph 2.3 M 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 274 k 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-3590.g7b710596.el9 ceph 4.9 M 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-3590.g7b710596.el9 ceph 18 M 2024-05-10T20:51:41.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3590.g7b710596.el9 ceph-noarch 16 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-3590.g7b710596.el9 ceph 25 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-3590.g7b710596.el9 ceph 157 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T20:51:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 497 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-3590.g7b710596.el9 ceph 5.2 M 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T20:51:41.302 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T20:51:41.303 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T20:51:41.304 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-3590.g7b710596.el9 ceph 45 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-3590.g7b710596.el9 ceph 135 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T20:51:41.305 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T20:51:41.306 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T20:51:41.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T20:51:41.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T20:51:41.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T20:51:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:51:41.311 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-10T20:51:41.312 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-10T20:51:41.312 INFO:teuthology.orchestra.run.smithi050.stdout:Install 148 Packages 2024-05-10T20:51:41.312 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 8 Packages 2024-05-10T20:51:41.312 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:51:41.312 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 219 M 2024-05-10T20:51:41.313 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-10T20:51:41.331 INFO:teuthology.orchestra.run.smithi092.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-10T20:51:41.332 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T20:51:41.363 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T20:51:41.391 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T20:51:41.425 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T20:51:41.473 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T20:51:41.507 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T20:51:41.534 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T20:51:41.569 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T20:51:41.601 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T20:51:41.643 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T20:51:41.677 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T20:51:41.765 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T20:51:41.824 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T20:51:41.839 INFO:teuthology.orchestra.run.smithi092.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-05-10T20:51:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T20:51:41.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T20:51:42.012 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T20:51:42.065 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T20:51:42.356 INFO:teuthology.orchestra.run.smithi092.stdout:(148/156): librados2-19.0.0-3590.g7b710596.el9. 3.2 MB/s | 3.3 MB 00:01 2024-05-10T20:51:42.731 INFO:teuthology.orchestra.run.smithi092.stdout:(149/156): librbd1-19.0.0-3590.g7b710596.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-05-10T20:51:42.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T20:51:42.837 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T20:51:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:(1/156): ceph-19.0.0-3590.g7b710596.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-05-10T20:51:42.871 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T20:51:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T20:51:43.020 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T20:51:43.031 INFO:teuthology.orchestra.run.smithi050.stdout:(2/156): ceph-fuse-19.0.0-3590.g7b710596.el9.x8 4.6 MB/s | 863 kB 00:00 2024-05-10T20:51:43.082 INFO:teuthology.orchestra.run.smithi050.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 2.9 MB/s | 147 kB 00:00 2024-05-10T20:51:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T20:51:43.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T20:51:43.274 INFO:teuthology.orchestra.run.smithi050.stdout:(4/156): ceph-mds-19.0.0-3590.g7b710596.el9.x86 12 MB/s | 2.3 MB 00:00 2024-05-10T20:51:43.315 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T20:51:43.345 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T20:51:43.390 INFO:teuthology.orchestra.run.smithi050.stdout:(5/156): ceph-mgr-19.0.0-3590.g7b710596.el9.x86 13 MB/s | 1.5 MB 00:00 2024-05-10T20:51:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T20:51:43.691 INFO:teuthology.orchestra.run.smithi050.stdout:(6/156): ceph-mon-19.0.0-3590.g7b710596.el9.x86 16 MB/s | 4.9 MB 00:00 2024-05-10T20:51:43.790 INFO:teuthology.orchestra.run.smithi092.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-10T20:51:43.857 INFO:teuthology.orchestra.run.smithi092.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2024-05-10T20:51:43.882 INFO:teuthology.orchestra.run.smithi092.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 455 kB/s | 42 kB 00:00 2024-05-10T20:51:43.949 INFO:teuthology.orchestra.run.smithi092.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.6 MB/s | 248 kB 00:00 2024-05-10T20:51:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T20:51:44.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T20:51:44.282 INFO:teuthology.orchestra.run.smithi092.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-05-10T20:51:44.407 INFO:teuthology.orchestra.run.smithi092.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 2.4 MB 00:00 2024-05-10T20:51:44.625 INFO:teuthology.orchestra.run.smithi050.stdout:(7/156): ceph-osd-19.0.0-3590.g7b710596.el9.x86 19 MB/s | 18 MB 00:00 2024-05-10T20:51:44.833 INFO:teuthology.orchestra.run.smithi050.stdout:(8/156): ceph-base-19.0.0-3590.g7b710596.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-10T20:51:44.925 INFO:teuthology.orchestra.run.smithi050.stdout:(9/156): ceph-selinux-19.0.0-3590.g7b710596.el9 278 kB/s | 25 kB 00:00 2024-05-10T20:51:45.159 INFO:teuthology.orchestra.run.smithi050.stdout:(10/156): ceph-radosgw-19.0.0-3590.g7b710596.el 19 MB/s | 10 MB 00:00 2024-05-10T20:51:45.217 INFO:teuthology.orchestra.run.smithi050.stdout:(11/156): cephfs-mirror-19.0.0-3590.g7b710596.e 3.8 MB/s | 224 kB 00:00 2024-05-10T20:51:45.267 INFO:teuthology.orchestra.run.smithi050.stdout:(12/156): cephfs-shell-19.0.0-3590.g7b710596.el 547 kB/s | 27 kB 00:00 2024-05-10T20:51:45.318 INFO:teuthology.orchestra.run.smithi050.stdout:(13/156): libcephfs-devel-19.0.0-3590.g7b710596 646 kB/s | 32 kB 00:00 2024-05-10T20:51:45.393 INFO:teuthology.orchestra.run.smithi050.stdout:(14/156): libcephfs2-19.0.0-3590.g7b710596.el9. 10 MB/s | 756 kB 00:00 2024-05-10T20:51:45.443 INFO:teuthology.orchestra.run.smithi050.stdout:(15/156): libcephsqlite-19.0.0-3590.g7b710596.e 3.1 MB/s | 157 kB 00:00 2024-05-10T20:51:45.494 INFO:teuthology.orchestra.run.smithi050.stdout:(16/156): librados-devel-19.0.0-3590.g7b710596. 2.5 MB/s | 127 kB 00:00 2024-05-10T20:51:45.495 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T20:51:45.552 INFO:teuthology.orchestra.run.smithi050.stdout:(17/156): libradosstriper1-19.0.0-3590.g7b71059 8.4 MB/s | 497 kB 00:00 2024-05-10T20:51:45.573 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T20:51:45.601 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T20:51:45.634 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T20:51:45.861 INFO:teuthology.orchestra.run.smithi050.stdout:(18/156): librgw2-19.0.0-3590.g7b710596.el9.x86 17 MB/s | 5.2 MB 00:00 2024-05-10T20:51:45.911 INFO:teuthology.orchestra.run.smithi050.stdout:(19/156): python3-ceph-argparse-19.0.0-3590.g7b 925 kB/s | 45 kB 00:00 2024-05-10T20:51:45.950 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T20:51:45.970 INFO:teuthology.orchestra.run.smithi050.stdout:(20/156): python3-ceph-common-19.0.0-3590.g7b71 2.3 MB/s | 135 kB 00:00 2024-05-10T20:51:45.997 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:51:46.028 INFO:teuthology.orchestra.run.smithi050.stdout:(21/156): python3-cephfs-19.0.0-3590.g7b710596. 2.8 MB/s | 162 kB 00:00 2024-05-10T20:51:46.071 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:51:46.087 INFO:teuthology.orchestra.run.smithi050.stdout:(22/156): python3-rados-19.0.0-3590.g7b710596.e 5.5 MB/s | 321 kB 00:00 2024-05-10T20:51:46.102 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 102/164 2024-05-10T20:51:46.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T20:51:46.146 INFO:teuthology.orchestra.run.smithi050.stdout:(23/156): python3-rbd-19.0.0-3590.g7b710596.el9 5.1 MB/s | 300 kB 00:00 2024-05-10T20:51:46.196 INFO:teuthology.orchestra.run.smithi050.stdout:(24/156): python3-rgw-19.0.0-3590.g7b710596.el9 2.0 MB/s | 99 kB 00:00 2024-05-10T20:51:46.247 INFO:teuthology.orchestra.run.smithi050.stdout:(25/156): rbd-fuse-19.0.0-3590.g7b710596.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-10T20:51:46.472 INFO:teuthology.orchestra.run.smithi050.stdout:(26/156): rbd-mirror-19.0.0-3590.g7b710596.el9. 14 MB/s | 3.0 MB 00:00 2024-05-10T20:51:46.522 INFO:teuthology.orchestra.run.smithi050.stdout:(27/156): rbd-nbd-19.0.0-3590.g7b710596.el9.x86 3.4 MB/s | 169 kB 00:00 2024-05-10T20:51:46.573 INFO:teuthology.orchestra.run.smithi050.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3590.g 697 kB/s | 34 kB 00:00 2024-05-10T20:51:46.623 INFO:teuthology.orchestra.run.smithi050.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3590.g7b71059 2.8 MB/s | 142 kB 00:00 2024-05-10T20:51:46.624 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T20:51:46.658 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:51:46.729 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:51:46.743 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 106/164 2024-05-10T20:51:46.882 INFO:teuthology.orchestra.run.smithi050.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3590.g7b710 14 MB/s | 3.5 MB 00:00 2024-05-10T20:51:47.359 INFO:teuthology.orchestra.run.smithi050.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-05-10T20:51:47.415 INFO:teuthology.orchestra.run.smithi050.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3590.g7b 4.8 MB/s | 274 kB 00:00 2024-05-10T20:51:47.466 INFO:teuthology.orchestra.run.smithi050.stdout:(33/156): ceph-mgr-rook-19.0.0-3590.g7b710596.e 1.0 MB/s | 49 kB 00:00 2024-05-10T20:51:47.508 INFO:teuthology.orchestra.run.smithi050.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3590.g7 405 kB/s | 16 kB 00:00 2024-05-10T20:51:47.566 INFO:teuthology.orchestra.run.smithi050.stdout:(35/156): ceph-volume-19.0.0-3590.g7b710596.el9 4.6 MB/s | 273 kB 00:00 2024-05-10T20:51:47.658 INFO:teuthology.orchestra.run.smithi050.stdout:(36/156): cephadm-19.0.0-3590.g7b710596.el9.noa 8.2 MB/s | 764 kB 00:00 2024-05-10T20:51:47.700 INFO:teuthology.orchestra.run.smithi050.stdout:(37/156): cephfs-top-19.0.0-3590.g7b710596.el9. 448 kB/s | 18 kB 00:00 2024-05-10T20:51:47.776 INFO:teuthology.orchestra.run.smithi050.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-05-10T20:51:47.884 INFO:teuthology.orchestra.run.smithi050.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-05-10T20:51:47.960 INFO:teuthology.orchestra.run.smithi050.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-05-10T20:51:48.043 INFO:teuthology.orchestra.run.smithi050.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-10T20:51:48.177 INFO:teuthology.orchestra.run.smithi050.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-05-10T20:51:48.253 INFO:teuthology.orchestra.run.smithi050.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-10T20:51:48.327 INFO:teuthology.orchestra.run.smithi050.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-10T20:51:48.411 INFO:teuthology.orchestra.run.smithi050.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-10T20:51:48.495 INFO:teuthology.orchestra.run.smithi050.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-10T20:51:48.925 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:51:48.972 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:51:49.270 INFO:teuthology.orchestra.run.smithi050.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 890 kB/s | 689 kB 00:00 2024-05-10T20:51:49.528 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:51:49.567 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:51:49.604 INFO:teuthology.orchestra.run.smithi050.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-10T20:51:49.663 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:51:49.663 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T20:51:49.663 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T20:51:49.663 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:51:49.693 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:51:49.729 INFO:teuthology.orchestra.run.smithi050.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 861 kB/s | 107 kB 00:00 2024-05-10T20:51:49.846 INFO:teuthology.orchestra.run.smithi050.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 186 kB/s | 21 kB 00:00 2024-05-10T20:51:49.963 INFO:teuthology.orchestra.run.smithi050.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-10T20:51:50.089 INFO:teuthology.orchestra.run.smithi050.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 33 kB 00:00 2024-05-10T20:51:50.589 INFO:teuthology.orchestra.run.smithi050.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-05-10T20:51:50.706 INFO:teuthology.orchestra.run.smithi050.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 158 kB/s | 18 kB 00:00 2024-05-10T20:51:50.806 INFO:teuthology.orchestra.run.smithi050.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 11 kB 00:00 2024-05-10T20:51:51.073 INFO:teuthology.orchestra.run.smithi050.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 602 kB/s | 160 kB 00:00 2024-05-10T20:51:51.182 INFO:teuthology.orchestra.run.smithi050.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 422 kB/s | 45 kB 00:00 2024-05-10T20:51:52.157 INFO:teuthology.orchestra.run.smithi050.stdout:(58/156): ceph-test-19.0.0-3590.g7b710596.el9.x 6.0 MB/s | 43 MB 00:07 2024-05-10T20:51:52.224 INFO:teuthology.orchestra.run.smithi050.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 636 kB/s | 662 kB 00:01 2024-05-10T20:51:52.324 INFO:teuthology.orchestra.run.smithi050.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 257 kB/s | 25 kB 00:00 2024-05-10T20:51:52.433 INFO:teuthology.orchestra.run.smithi050.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 442 kB/s | 47 kB 00:00 2024-05-10T20:51:52.567 INFO:teuthology.orchestra.run.smithi050.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 292 kB 00:00 2024-05-10T20:51:52.834 INFO:teuthology.orchestra.run.smithi050.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 709 kB/s | 188 kB 00:00 2024-05-10T20:51:52.942 INFO:teuthology.orchestra.run.smithi050.stdout:(64/156): libtool-2.4.6-45.el9.x86_64.rpm 743 kB/s | 582 kB 00:00 2024-05-10T20:51:52.968 INFO:teuthology.orchestra.run.smithi050.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 265 kB/s | 35 kB 00:00 2024-05-10T20:51:53.818 INFO:teuthology.orchestra.run.smithi050.stdout:(66/156): ceph-common-19.0.0-3590.g7b710596.el9 1.7 MB/s | 20 MB 00:11 2024-05-10T20:51:54.819 INFO:teuthology.orchestra.run.smithi050.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-05-10T20:51:54.960 INFO:teuthology.orchestra.run.smithi050.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-10T20:51:55.077 INFO:teuthology.orchestra.run.smithi050.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-05-10T20:51:55.111 INFO:teuthology.orchestra.run.smithi050.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 890 kB/s | 133 kB 00:00 2024-05-10T20:51:55.245 INFO:teuthology.orchestra.run.smithi050.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-05-10T20:51:55.353 INFO:teuthology.orchestra.run.smithi050.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 813 kB/s | 87 kB 00:00 2024-05-10T20:51:55.779 INFO:teuthology.orchestra.run.smithi050.stdout:(73/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.7 MB/s | 4.7 MB 00:02 2024-05-10T20:51:56.004 INFO:teuthology.orchestra.run.smithi050.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-05-10T20:51:56.121 INFO:teuthology.orchestra.run.smithi050.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-10T20:51:56.171 INFO:teuthology.orchestra.run.smithi050.stdout:(76/156): python3-devel-3.9.18-3.el9.x86_64.rpm 624 kB/s | 244 kB 00:00 2024-05-10T20:51:56.230 INFO:teuthology.orchestra.run.smithi050.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-05-10T20:51:56.280 INFO:teuthology.orchestra.run.smithi050.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-10T20:51:56.339 INFO:teuthology.orchestra.run.smithi050.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-10T20:51:56.389 INFO:teuthology.orchestra.run.smithi050.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-10T20:51:56.748 INFO:teuthology.orchestra.run.smithi050.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-10T20:51:56.881 INFO:teuthology.orchestra.run.smithi050.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 5.1 MB 00:00 2024-05-10T20:51:56.990 INFO:teuthology.orchestra.run.smithi050.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 718 kB/s | 77 kB 00:00 2024-05-10T20:51:57.098 INFO:teuthology.orchestra.run.smithi050.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 393 kB/s | 42 kB 00:00 2024-05-10T20:51:57.140 INFO:teuthology.orchestra.run.smithi050.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 567 kB/s | 222 kB 00:00 2024-05-10T20:51:57.216 INFO:teuthology.orchestra.run.smithi050.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-05-10T20:51:57.333 INFO:teuthology.orchestra.run.smithi050.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-10T20:51:57.433 INFO:teuthology.orchestra.run.smithi050.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-05-10T20:51:57.458 INFO:teuthology.orchestra.run.smithi050.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 410 kB/s | 51 kB 00:00 2024-05-10T20:51:57.542 INFO:teuthology.orchestra.run.smithi050.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 497 kB/s | 54 kB 00:00 2024-05-10T20:51:57.659 INFO:teuthology.orchestra.run.smithi050.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 358 kB/s | 42 kB 00:00 2024-05-10T20:51:57.776 INFO:teuthology.orchestra.run.smithi050.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 381 kB/s | 44 kB 00:00 2024-05-10T20:51:58.151 INFO:teuthology.orchestra.run.smithi050.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 590 kB/s | 221 kB 00:00 2024-05-10T20:51:58.260 INFO:teuthology.orchestra.run.smithi050.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 333 kB/s | 36 kB 00:00 2024-05-10T20:51:58.368 INFO:teuthology.orchestra.run.smithi050.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-10T20:51:58.839 INFO:teuthology.orchestra.run.smithi092.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 727 kB/s | 16 MB 00:22 2024-05-10T20:51:58.842 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-10T20:51:58.842 INFO:teuthology.orchestra.run.smithi092.stdout:Total 5.0 MB/s | 219 MB 00:44 2024-05-10T20:51:58.944 INFO:teuthology.orchestra.run.smithi050.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-10T20:51:58.977 INFO:teuthology.orchestra.run.smithi050.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 501 kB/s | 305 kB 00:00 2024-05-10T20:51:59.053 INFO:teuthology.orchestra.run.smithi050.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 537 kB/s | 58 kB 00:00 2024-05-10T20:51:59.161 INFO:teuthology.orchestra.run.smithi050.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 591 kB/s | 64 kB 00:00 2024-05-10T20:51:59.187 INFO:teuthology.orchestra.run.smithi050.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 449 kB/s | 94 kB 00:00 2024-05-10T20:51:59.271 INFO:teuthology.orchestra.run.smithi050.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-05-10T20:51:59.296 INFO:teuthology.orchestra.run.smithi050.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-05-10T20:51:59.638 INFO:teuthology.orchestra.run.smithi050.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-05-10T20:51:59.813 INFO:teuthology.orchestra.run.smithi050.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-10T20:51:59.913 INFO:teuthology.orchestra.run.smithi050.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 201 kB/s | 19 kB 00:00 2024-05-10T20:52:00.004 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-10T20:52:00.153 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-10T20:52:00.153 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-10T20:52:00.155 INFO:teuthology.orchestra.run.smithi050.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-05-10T20:52:00.285 INFO:teuthology.orchestra.run.smithi050.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 203 kB/s | 26 kB 00:00 2024-05-10T20:52:00.389 INFO:teuthology.orchestra.run.smithi050.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-05-10T20:52:00.414 INFO:teuthology.orchestra.run.smithi050.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 378 kB/s | 49 kB 00:00 2024-05-10T20:52:00.473 INFO:teuthology.orchestra.run.smithi050.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 816 kB/s | 67 kB 00:00 2024-05-10T20:52:00.523 INFO:teuthology.orchestra.run.smithi050.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-10T20:52:00.607 INFO:teuthology.orchestra.run.smithi050.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-05-10T20:52:00.690 INFO:teuthology.orchestra.run.smithi050.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 529 kB/s | 43 kB 00:00 2024-05-10T20:52:00.724 INFO:teuthology.orchestra.run.smithi050.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-10T20:52:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 390 kB/s | 32 kB 00:00 2024-05-10T20:52:00.808 INFO:teuthology.orchestra.run.smithi050.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-05-10T20:52:00.858 INFO:teuthology.orchestra.run.smithi050.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-10T20:52:00.909 INFO:teuthology.orchestra.run.smithi050.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-10T20:52:00.959 INFO:teuthology.orchestra.run.smithi050.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-10T20:52:01.009 INFO:teuthology.orchestra.run.smithi050.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 355 kB/s | 35 kB 00:00 2024-05-10T20:52:01.068 INFO:teuthology.orchestra.run.smithi050.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-05-10T20:52:01.152 INFO:teuthology.orchestra.run.smithi050.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-10T20:52:01.199 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-05-10T20:52:01.200 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:01.235 INFO:teuthology.orchestra.run.smithi050.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.1 MB/s | 2.0 MB 00:00 2024-05-10T20:52:01.260 INFO:teuthology.orchestra.run.smithi050.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-10T20:52:01.311 INFO:teuthology.orchestra.run.smithi050.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 238 kB/s | 18 kB 00:00 2024-05-10T20:52:01.336 INFO:teuthology.orchestra.run.smithi050.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 311 kB/s | 23 kB 00:00 2024-05-10T20:52:01.386 INFO:teuthology.orchestra.run.smithi050.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-05-10T20:52:01.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:01.411 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:01.411 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:01.411 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T20:52:01.412 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:01.420 INFO:teuthology.orchestra.run.smithi050.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-05-10T20:52:01.512 INFO:teuthology.orchestra.run.smithi050.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-05-10T20:52:01.538 INFO:teuthology.orchestra.run.smithi050.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-05-10T20:52:01.596 INFO:teuthology.orchestra.run.smithi050.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 955 kB/s | 79 kB 00:00 2024-05-10T20:52:01.621 INFO:teuthology.orchestra.run.smithi050.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 697 kB/s | 58 kB 00:00 2024-05-10T20:52:01.705 INFO:teuthology.orchestra.run.smithi050.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-10T20:52:01.730 INFO:teuthology.orchestra.run.smithi050.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-10T20:52:01.781 INFO:teuthology.orchestra.run.smithi050.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-10T20:52:01.823 INFO:teuthology.orchestra.run.smithi050.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 309 kB/s | 28 kB 00:00 2024-05-10T20:52:01.856 INFO:teuthology.orchestra.run.smithi050.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 414 kB/s | 31 kB 00:00 2024-05-10T20:52:01.898 INFO:teuthology.orchestra.run.smithi050.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-05-10T20:52:01.940 INFO:teuthology.orchestra.run.smithi050.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-05-10T20:52:01.982 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-10T20:52:01.982 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-10T20:52:02.024 INFO:teuthology.orchestra.run.smithi050.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2024-05-10T20:52:02.074 INFO:teuthology.orchestra.run.smithi050.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 385 kB/s | 51 kB 00:00 2024-05-10T20:52:02.133 INFO:teuthology.orchestra.run.smithi050.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-10T20:52:02.158 INFO:teuthology.orchestra.run.smithi050.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-10T20:52:02.217 INFO:teuthology.orchestra.run.smithi050.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.1 MB/s | 427 kB 00:00 2024-05-10T20:52:02.242 INFO:teuthology.orchestra.run.smithi050.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-05-10T20:52:02.309 INFO:teuthology.orchestra.run.smithi050.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-05-10T20:52:02.401 INFO:teuthology.orchestra.run.smithi050.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-10T20:52:02.543 INFO:teuthology.orchestra.run.smithi050.stdout:(148/156): postgresql-server-13.14-1.el9.x86_64 793 kB/s | 5.8 MB 00:07 2024-05-10T20:52:02.777 INFO:teuthology.orchestra.run.smithi050.stdout:(149/156): librados2-19.0.0-3590.g7b710596.el9. 7.1 MB/s | 3.3 MB 00:00 2024-05-10T20:52:02.975 INFO:teuthology.orchestra.run.smithi050.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 9.3 MB/s | 1.8 MB 00:00 2024-05-10T20:52:02.990 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:03.016 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:03.016 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:03.016 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T20:52:03.016 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:03.016 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:03.017 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:03.044 INFO:teuthology.orchestra.run.smithi050.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-05-10T20:52:03.169 INFO:teuthology.orchestra.run.smithi050.stdout:(152/156): librbd1-19.0.0-3590.g7b710596.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-05-10T20:52:03.194 INFO:teuthology.orchestra.run.smithi050.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 192 kB/s | 42 kB 00:00 2024-05-10T20:52:03.219 INFO:teuthology.orchestra.run.smithi050.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.4 MB/s | 248 kB 00:00 2024-05-10T20:52:03.252 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:03.275 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:03.353 INFO:teuthology.orchestra.run.smithi050.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-10T20:52:03.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:03.768 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:03.875 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T20:52:03.877 INFO:teuthology.orchestra.run.smithi050.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-05-10T20:52:03.889 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-10T20:52:03.889 INFO:teuthology.orchestra.run.smithi050.stdout:Total 9.7 MB/s | 219 MB 00:22 2024-05-10T20:52:03.932 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T20:52:03.987 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T20:52:04.174 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T20:52:04.221 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T20:52:04.298 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T20:52:04.387 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T20:52:04.526 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T20:52:04.579 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T20:52:04.611 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T20:52:04.655 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T20:52:05.079 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:05.112 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:05.144 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 126/164 2024-05-10T20:52:05.206 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 127/164 2024-05-10T20:52:05.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:06.265 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:06.295 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:06.967 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-10T20:52:07.227 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-10T20:52:07.227 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-10T20:52:07.253 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:07.287 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:07.398 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:07.561 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 131/164 2024-05-10T20:52:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:07.620 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:07.621 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:07.621 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T20:52:07.621 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:07.621 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:07.621 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:07.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:07.676 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:07.774 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 134/164 2024-05-10T20:52:07.829 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 135/164 2024-05-10T20:52:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-10T20:52:08.092 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T20:52:08.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T20:52:08.184 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T20:52:08.218 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T20:52:08.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T20:52:08.287 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T20:52:08.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T20:52:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:08.762 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T20:52:08.792 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:08.815 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:08.815 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:08.815 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T20:52:08.815 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:08.816 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:08.816 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:08.859 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:08.881 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:08.881 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:08.882 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T20:52:08.882 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:08.894 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:08.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:08.950 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:09.012 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:09.041 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:52:09.192 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T20:52:09.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T20:52:09.261 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:52:09.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:09.284 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:52:09.284 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:09.284 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T20:52:09.284 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:52:09.285 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:52:09.285 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:09.371 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:09.401 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:52:09.463 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:52:09.485 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:52:09.503 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:52:09.660 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:52:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:52:09.860 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-10T20:52:09.860 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-10T20:52:09.941 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:52:10.001 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:52:10.122 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:10.186 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:10.378 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:10.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T20:52:10.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T20:52:10.566 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-10T20:52:10.640 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-10T20:52:10.682 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-10T20:52:10.726 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-10T20:52:10.758 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-10T20:52:10.795 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-10T20:52:10.864 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-10T20:52:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-10T20:52:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-10T20:52:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-10T20:52:11.059 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-10T20:52:11.094 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 32/164 2024-05-10T20:52:11.137 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 33/164 2024-05-10T20:52:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 34/164 2024-05-10T20:52:11.236 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-10T20:52:11.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-10T20:52:11.364 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-10T20:52:11.516 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-10T20:52:11.575 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-10T20:52:12.120 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-10T20:52:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-10T20:52:12.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-10T20:52:12.284 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-10T20:52:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 44/164 2024-05-10T20:52:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-10T20:52:12.532 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-10T20:52:12.591 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-10T20:52:12.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-10T20:52:12.755 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-10T20:52:12.828 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T20:52:12.861 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T20:52:12.896 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T20:52:12.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T20:52:12.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T20:52:13.022 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T20:52:13.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T20:52:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T20:52:13.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T20:52:13.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T20:52:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T20:52:13.417 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T20:52:13.449 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T20:52:13.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T20:52:13.506 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T20:52:13.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T20:52:13.539 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 141/164 2024-05-10T20:52:13.589 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 142/164 2024-05-10T20:52:13.605 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T20:52:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 143/164 2024-05-10T20:52:13.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T20:52:13.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T20:52:13.732 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T20:52:13.763 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T20:52:13.808 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T20:52:13.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T20:52:13.869 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 146/164 2024-05-10T20:52:13.889 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T20:52:13.918 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T20:52:13.929 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T20:52:13.949 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 148/164 2024-05-10T20:52:13.968 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T20:52:13.978 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T20:52:14.002 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T20:52:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T20:52:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T20:52:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T20:52:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T20:52:14.198 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T20:52:14.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T20:52:14.281 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T20:52:14.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T20:52:14.434 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T20:52:14.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T20:52:14.679 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:14.714 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:14.733 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T20:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T20:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T20:52:15.415 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T20:52:15.472 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T20:52:15.558 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 152/164 2024-05-10T20:52:15.559 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T20:52:15.608 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T20:52:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T20:52:15.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T20:52:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T20:52:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T20:52:15.805 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T20:52:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T20:52:15.831 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T20:52:15.831 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:52:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T20:52:15.915 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:52:15.948 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T20:52:15.948 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:52:16.016 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:52:16.041 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T20:52:16.046 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:16.046 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:16.234 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:16.234 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:52:16.327 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:52:16.389 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:16.536 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T20:52:16.568 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-10T20:52:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T20:52:16.661 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T20:52:16.733 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T20:52:16.831 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T20:52:16.889 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T20:52:16.921 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T20:52:17.025 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T20:52:17.475 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T20:52:17.586 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:17.772 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:17.803 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T20:52:17.946 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:17.951 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T20:52:17.978 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:52:17.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T20:52:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T20:52:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T20:52:18.327 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T20:52:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T20:52:18.374 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:52:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:18.465 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:52:18.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 102/164 2024-05-10T20:52:18.534 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T20:52:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:52:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 14/164 2024-05-10T20:52:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:52:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T20:52:19.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T20:52:19.054 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:52:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:52:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:52:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 106/164 2024-05-10T20:52:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T20:52:19.566 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:52:19.654 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T20:52:19.878 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:20.011 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:20.364 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T20:52:20.572 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T20:52:20.618 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T20:52:20.676 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-10T20:52:20.797 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-10T20:52:20.848 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-10T20:52:20.925 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-10T20:52:21.003 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-10T20:52:21.042 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-10T20:52:21.169 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-10T20:52:21.247 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-10T20:52:21.284 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-10T20:52:21.341 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-10T20:52:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:21.452 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-10T20:52:21.487 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 32/164 2024-05-10T20:52:21.543 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 33/164 2024-05-10T20:52:21.578 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 34/164 2024-05-10T20:52:21.703 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-10T20:52:21.764 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-10T20:52:21.867 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-10T20:52:21.892 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:21.892 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 1/164 2024-05-10T20:52:21.892 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2/164 2024-05-10T20:52:21.892 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 3/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 4/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 5/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 6/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 7/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 11/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 12/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:21.893 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 14/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 15/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 16/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 17/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 18/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 19/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 20/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 21/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 22/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 23/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 24/164 2024-05-10T20:52:21.894 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 25/164 2024-05-10T20:52:21.895 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 26/164 2024-05-10T20:52:21.895 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 27/164 2024-05-10T20:52:21.895 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 28/164 2024-05-10T20:52:21.895 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 29/164 2024-05-10T20:52:21.895 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 30/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 31/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 32/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 33/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 34/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 35/164 2024-05-10T20:52:21.909 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 36/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 37/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 38/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 39/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T20:52:21.910 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T20:52:21.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T20:52:21.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T20:52:21.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T20:52:21.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T20:52:21.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T20:52:21.916 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T20:52:21.917 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T20:52:21.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T20:52:21.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 149/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 151/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T20:52:21.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T20:52:21.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T20:52:21.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T20:52:21.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T20:52:21.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T20:52:21.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T20:52:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:52:22.060 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-10T20:52:22.094 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:52:22.094 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T20:52:22.095 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T20:52:22.095 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:22.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:52:22.127 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-10T20:52:22.864 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-10T20:52:22.977 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-10T20:52:23.018 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-10T20:52:23.063 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-10T20:52:23.104 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 44/164 2024-05-10T20:52:23.164 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-10T20:52:23.379 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-10T20:52:23.436 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-10T20:52:23.563 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-10T20:52:23.632 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-10T20:52:23.728 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T20:52:23.771 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T20:52:23.808 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T20:52:23.874 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T20:52:23.918 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T20:52:23.976 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T20:52:24.047 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T20:52:24.251 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T20:52:24.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T20:52:24.345 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T20:52:24.391 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T20:52:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T20:52:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T20:52:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T20:52:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T20:52:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.725 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.726 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.727 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T20:52:24.728 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T20:52:24.729 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T20:52:24.730 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.731 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T20:52:24.732 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T20:52:24.733 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T20:52:24.734 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T20:52:24.735 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T20:52:24.736 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T20:52:24.737 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T20:52:24.737 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T20:52:24.737 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:52:24.737 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-05-10T20:52:24.840 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T20:52:25.030 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T20:52:25.075 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T20:52:25.114 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T20:52:25.165 DEBUG:teuthology.parallel:result is None 2024-05-10T20:52:25.181 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T20:52:25.248 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T20:52:25.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T20:52:25.352 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T20:52:25.391 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T20:52:25.450 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T20:52:25.528 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T20:52:25.576 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T20:52:25.722 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T20:52:25.786 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T20:52:25.831 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T20:52:25.903 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T20:52:26.050 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T20:52:26.128 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T20:52:27.132 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T20:52:27.185 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T20:52:27.249 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T20:52:27.391 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T20:52:27.457 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T20:52:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T20:52:27.612 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T20:52:27.858 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T20:52:27.891 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T20:52:28.000 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T20:52:28.890 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T20:52:29.052 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T20:52:30.761 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T20:52:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T20:52:30.912 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T20:52:30.954 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T20:52:31.359 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T20:52:31.402 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:52:31.509 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 101/164 2024-05-10T20:52:31.542 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 102/164 2024-05-10T20:52:31.599 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T20:52:32.206 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T20:52:32.239 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:52:32.340 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 105/164 2024-05-10T20:52:32.374 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 106/164 2024-05-10T20:52:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:52:33.597 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-05-10T20:52:33.597 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-05-10T20:52:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:33.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:33.783 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:33.783 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T20:52:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:35.168 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:35.214 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:35.354 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:35.380 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:35.605 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:35.627 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:35.627 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:35.627 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T20:52:35.627 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:35.628 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:35.628 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:35.925 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 107/164 2024-05-10T20:52:35.977 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:52:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:36.177 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 108/164 2024-05-10T20:52:36.177 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T20:52:36.177 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T20:52:36.177 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:36.211 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:52:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T20:52:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T20:52:36.324 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T20:52:36.509 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T20:52:36.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T20:52:36.635 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T20:52:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T20:52:36.865 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T20:52:36.907 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T20:52:36.948 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T20:52:36.984 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T20:52:37.432 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:37.458 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 126/164 2024-05-10T20:52:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 127/164 2024-05-10T20:52:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:39.624 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:39.658 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:39.769 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:39.941 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 131/164 2024-05-10T20:52:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:39.999 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:40.000 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T20:52:40.000 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:40.000 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:40.153 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 134/164 2024-05-10T20:52:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 135/164 2024-05-10T20:52:41.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T20:52:41.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:41.202 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:41.203 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T20:52:41.203 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:41.203 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:41.269 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:41.269 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T20:52:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:41.650 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:52:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:52:41.671 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:41.671 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T20:52:41.671 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:52:41.672 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:52:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 141/164 2024-05-10T20:52:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 142/164 2024-05-10T20:52:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 143/164 2024-05-10T20:52:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T20:52:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T20:52:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 146/164 2024-05-10T20:52:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T20:52:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 148/164 2024-05-10T20:52:46.384 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T20:52:46.422 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T20:52:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:47.023 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:52:47.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 152/164 2024-05-10T20:52:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T20:52:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T20:52:48.056 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T20:52:48.119 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T20:52:48.145 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T20:52:48.146 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:52:48.230 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:52:48.254 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T20:52:48.255 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:52:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:52:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T20:52:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:48.541 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:52:48.541 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:52:48.625 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:52:48.687 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:49.822 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 109/164 2024-05-10T20:52:49.822 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-05-10T20:52:49.823 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:50.071 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:50.120 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 110/164 2024-05-10T20:52:50.120 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:50.120 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T20:52:50.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:51.931 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:52.036 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 111/164 2024-05-10T20:52:52.037 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:52.037 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T20:52:52.037 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:52.037 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T20:52:52.037 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:52.307 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:52.404 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 112/164 2024-05-10T20:52:52.405 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:52.405 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T20:52:52.405 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:52.405 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T20:52:52.405 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:52.941 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 113/164 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T20:52:53.039 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:53.175 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T20:52:53.238 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T20:52:53.312 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T20:52:53.569 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T20:52:53.622 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T20:52:53.711 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T20:52:53.830 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T20:52:54.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T20:52:54.065 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T20:52:54.112 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T20:52:54.144 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 1/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 3/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 4/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 5/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 6/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 7/164 2024-05-10T20:52:54.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 11/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 12/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 14/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 15/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 16/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 17/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 18/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 19/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 20/164 2024-05-10T20:52:54.348 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 21/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 22/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 23/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 24/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 25/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 26/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 27/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 28/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 29/164 2024-05-10T20:52:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 30/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 31/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 32/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 33/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 34/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 35/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 36/164 2024-05-10T20:52:54.362 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 37/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 38/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 39/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T20:52:54.363 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T20:52:54.364 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T20:52:54.365 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T20:52:54.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T20:52:54.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T20:52:54.368 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T20:52:54.369 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T20:52:54.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 149/164 2024-05-10T20:52:54.371 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 151/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T20:52:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T20:52:54.756 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:54.790 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 125/164 2024-05-10T20:52:54.847 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 126/164 2024-05-10T20:52:54.945 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 127/164 2024-05-10T20:52:54.982 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:56.331 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 128/164 2024-05-10T20:52:56.377 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T20:52:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T20:52:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T20:52:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T20:52:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T20:52:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T20:52:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T20:52:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T20:52:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T20:52:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-10T20:52:56.959 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-10T20:52:57.349 DEBUG:teuthology.parallel:result is None 2024-05-10T20:52:57.542 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 129/164 2024-05-10T20:52:57.568 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:57.734 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 130/164 2024-05-10T20:52:57.922 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 131/164 2024-05-10T20:52:57.962 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:58.055 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 132/164 2024-05-10T20:52:58.055 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:58.056 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T20:52:58.056 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:58.056 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T20:52:58.056 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:58.113 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:58.153 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 133/164 2024-05-10T20:52:58.270 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 134/164 2024-05-10T20:52:58.345 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 135/164 2024-05-10T20:52:59.429 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T20:52:59.465 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 137/164 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T20:52:59.556 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:59.607 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:59.702 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3590.g7b710 138/164 2024-05-10T20:52:59.702 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:59.702 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T20:52:59.702 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:52:59.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:59.866 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 139/164 2024-05-10T20:52:59.866 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:52:59.866 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T20:52:59.866 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:59.867 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T20:52:59.867 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:53:00.284 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:53:00.357 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 140/164 2024-05-10T20:53:00.358 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T20:53:00.358 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T20:53:00.358 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:53:00.358 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T20:53:00.358 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:53:05.315 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 141/164 2024-05-10T20:53:05.368 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 142/164 2024-05-10T20:53:05.431 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 143/164 2024-05-10T20:53:05.464 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T20:53:05.502 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T20:53:05.842 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 146/164 2024-05-10T20:53:05.903 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T20:53:05.932 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 148/164 2024-05-10T20:53:05.977 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T20:53:06.042 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T20:53:06.955 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:53:07.014 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:53:07.075 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T20:53:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 152/164 2024-05-10T20:53:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T20:53:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T20:53:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T20:53:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T20:53:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T20:53:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:53:09.115 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T20:53:09.161 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T20:53:09.161 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:53:09.287 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T20:53:09.320 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T20:53:09.353 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:53:09.353 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:53:09.687 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T20:53:09.688 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:53:09.820 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T20:53:09.954 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:53:10.226 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:53:10.226 INFO:teuthology.orchestra.run.smithi050.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-10T20:53:10.226 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:53:16.168 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-3590.g7b710596.el9.x86_64 1/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 3/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 4/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3590.g7b710 5/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 6/164 2024-05-10T20:53:16.169 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 7/164 2024-05-10T20:53:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 8/164 2024-05-10T20:53:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 9/164 2024-05-10T20:53:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 10/164 2024-05-10T20:53:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 11/164 2024-05-10T20:53:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 12/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 13/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 14/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_ 15/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 16/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 17/164 2024-05-10T20:53:16.171 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-3590.g7b710596.el9.x86_6 18/164 2024-05-10T20:53:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86 19/164 2024-05-10T20:53:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 20/164 2024-05-10T20:53:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3590.g7b710596.el 21/164 2024-05-10T20:53:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-3590.g7b710596.el9. 22/164 2024-05-10T20:53:16.172 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_6 23/164 2024-05-10T20:53:16.173 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 24/164 2024-05-10T20:53:16.173 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 25/164 2024-05-10T20:53:16.173 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 26/164 2024-05-10T20:53:16.173 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 27/164 2024-05-10T20:53:16.174 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 28/164 2024-05-10T20:53:16.174 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 29/164 2024-05-10T20:53:16.174 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3590.g7b710596. 30/164 2024-05-10T20:53:16.174 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noa 31/164 2024-05-10T20:53:16.174 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.n 32/164 2024-05-10T20:53:16.175 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b7 33/164 2024-05-10T20:53:16.175 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el 34/164 2024-05-10T20:53:16.175 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 35/164 2024-05-10T20:53:16.175 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.e 36/164 2024-05-10T20:53:16.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 37/164 2024-05-10T20:53:16.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-3590.g7b710596.el9.noarch 38/164 2024-05-10T20:53:16.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 39/164 2024-05-10T20:53:16.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T20:53:16.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T20:53:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T20:53:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T20:53:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T20:53:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T20:53:16.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T20:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T20:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T20:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T20:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T20:53:16.178 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T20:53:16.179 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T20:53:16.179 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T20:53:16.179 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T20:53:16.179 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T20:53:16.180 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T20:53:16.180 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T20:53:16.180 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T20:53:16.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T20:53:16.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T20:53:16.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T20:53:16.182 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T20:53:16.182 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T20:53:16.182 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T20:53:16.183 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T20:53:16.184 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T20:53:16.185 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T20:53:16.186 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T20:53:16.186 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T20:53:16.187 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T20:53:16.188 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T20:53:16.188 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T20:53:16.188 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T20:53:16.188 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T20:53:16.188 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T20:53:16.189 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T20:53:16.190 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T20:53:16.190 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T20:53:16.190 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T20:53:16.190 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T20:53:16.190 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T20:53:16.191 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T20:53:16.192 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T20:53:16.193 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T20:53:16.194 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T20:53:16.194 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T20:53:16.194 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T20:53:16.194 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T20:53:16.194 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T20:53:16.195 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T20:53:16.196 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T20:53:16.196 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T20:53:16.196 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T20:53:16.196 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T20:53:16.197 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T20:53:16.197 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T20:53:16.197 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T20:53:16.197 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T20:53:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T20:53:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T20:53:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T20:53:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T20:53:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T20:53:16.199 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-3590.g7b710596.el9.x86_64 149/164 2024-05-10T20:53:16.199 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T20:53:16.199 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 151/164 2024-05-10T20:53:16.199 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T20:53:16.200 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T20:53:16.201 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T20:53:16.201 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T20:53:16.201 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T20:53:16.202 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T20:53:16.202 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T20:53:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-3590.g7b710596.el9.noarch 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T20:53:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T20:53:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T20:53:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T20:53:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T20:53:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-3590.g7b710596.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T20:53:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T20:53:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T20:53:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:53:18.386 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-10T20:53:19.004 DEBUG:teuthology.parallel:result is None 2024-05-10T20:53:19.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:19.150 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T20:53:19.208 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.209 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.209 INFO:teuthology.task.install:The correct ceph version 19.0.0-3590.g7b710596 is installed. 2024-05-10T20:53:19.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:19.357 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T20:53:19.382 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.383 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.383 INFO:teuthology.task.install:The correct ceph version 19.0.0-3590.g7b710596 is installed. 2024-05-10T20:53:19.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:19.522 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T20:53:19.548 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.548 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3590.g7b710596.el9 2024-05-10T20:53:19.548 INFO:teuthology.task.install:The correct ceph version 19.0.0-3590.g7b710596 is installed. 2024-05-10T20:53:19.550 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T20:53:19.550 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:53:19.550 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T20:53:19.620 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:53:19.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T20:53:19.650 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:53:19.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T20:53:19.680 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T20:53:19.681 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:53:19.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T20:53:19.748 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T20:53:19.866 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:53:19.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T20:53:19.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T20:53:19.961 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:53:19.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T20:53:19.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T20:53:20.054 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T20:53:20.055 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:53:20.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T20:53:20.128 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T20:53:20.249 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:53:20.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T20:53:20.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T20:53:20.345 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:53:20.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T20:53:20.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T20:53:20.434 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T20:53:20.434 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:53:20.434 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T20:53:20.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T20:53:20.607 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:53:20.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T20:53:20.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T20:53:20.700 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:53:20.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T20:53:20.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T20:53:20.787 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T20:53:20.896 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-10T20:53:20.897 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:20.897 INFO:tasks.cephadm:Cluster fsid is 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:53:20.897 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T20:53:20.897 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.50', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.142'} 2024-05-10T20:53:20.897 INFO:tasks.cephadm:First mon is mon.a on smithi050 2024-05-10T20:53:20.897 INFO:tasks.cephadm:First mgr is x 2024-05-10T20:53:20.898 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T20:53:20.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-05-10T20:53:20.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-05-10T20:53:20.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-05-10T20:53:21.022 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T20:53:21.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:21.164 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79248/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3590-g7b710596', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3590.g7b710596'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 14:51:14.136120', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.114851-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '7b710596db3f44de0a4b72eff459db857656fe05', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79246/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3590-g7b710596', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3590.g7b710596'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 14:39:11.833658', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.114851-debug', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T20:53:21.411 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240509.114851-debug, sha1 7b710596db3f44de0a4b72eff459db857656fe05 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:53:21.413 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/x86_64/flavors/default/cephadm 2024-05-10T20:53:21.413 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:22.085 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 782908 May 10 20:53 /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:22.085 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:22.779 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 782908 May 10 20:53 /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:22.779 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.114851-debug/7b710596db3f44de0a4b72eff459db857656fe05/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:23.505 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 782908 May 10 20:53 /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:23.506 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:23.558 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:23.576 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T20:53:23.615 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 on all hosts... 2024-05-10T20:53:23.615 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 pull 2024-05-10T20:53:23.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 pull 2024-05-10T20:53:23.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 pull 2024-05-10T20:53:25.194 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05... 2024-05-10T20:53:25.239 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05... 2024-05-10T20:53:26.387 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05... 2024-05-10T20:54:13.476 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T20:54:13.476 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev)", 2024-05-10T20:54:13.476 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62", 2024-05-10T20:54:13.476 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-05-10T20:54:13.477 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95" 2024-05-10T20:54:13.477 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T20:54:13.477 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T20:54:18.735 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev)", 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62", 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95" 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-05-10T20:54:18.736 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-10T20:54:23.324 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev)", 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62", 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95" 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-05-10T20:54:23.325 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T20:54:23.353 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-05-10T20:54:23.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-05-10T20:54:23.483 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-05-10T20:54:23.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-05-10T20:54:23.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-05-10T20:54:23.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-05-10T20:54:23.656 INFO:tasks.cephadm:Writing seed config... 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T20:54:23.658 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T20:54:23.659 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T20:54:23.660 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T20:54:23.661 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T20:54:23.661 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T20:54:23.661 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T20:54:23.661 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T20:54:23.661 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-10T20:54:23.662 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:54:23.662 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T20:54:23.722 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 55ba4db6-0f0f-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T20:54:23.722 DEBUG:teuthology.orchestra.run.smithi050:mon.a> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T20:54:23.770 DEBUG:teuthology.orchestra.run.smithi050:mgr.x> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T20:54:23.816 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T20:54:23.816 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 -v bootstrap --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.50 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T20:54:24.090 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-10T20:54:24.090 INFO:teuthology.orchestra.run.smithi050.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05', '-v', 'bootstrap', '--fsid', '55ba4db6-0f0f-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.50', '--skip-admin-label'] 2024-05-10T20:54:24.090 INFO:teuthology.orchestra.run.smithi050.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T20:54:24.090 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying podman|docker is present... 2024-05-10T20:54:24.151 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T20:54:24.151 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying lvm2 is present... 2024-05-10T20:54:24.151 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying time synchronization is in place... 2024-05-10T20:54:24.168 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T20:54:24.168 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T20:54:24.184 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T20:54:24.184 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:54:24.206 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-05-10T20:54:24.227 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-05-10T20:54:24.227 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-05-10T20:54:24.228 INFO:teuthology.orchestra.run.smithi050.stdout:Repeating the final host check... 2024-05-10T20:54:24.290 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T20:54:24.291 INFO:teuthology.orchestra.run.smithi050.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-10T20:54:24.291 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl is present 2024-05-10T20:54:24.291 INFO:teuthology.orchestra.run.smithi050.stdout:lvcreate is present 2024-05-10T20:54:24.308 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T20:54:24.308 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T20:54:24.326 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T20:54:24.327 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:54:24.350 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-05-10T20:54:24.371 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-05-10T20:54:24.371 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-05-10T20:54:24.372 INFO:teuthology.orchestra.run.smithi050.stdout:Host looks OK 2024-05-10T20:54:24.372 INFO:teuthology.orchestra.run.smithi050.stdout:Cluster fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:24.372 INFO:teuthology.orchestra.run.smithi050.stdout:Acquiring lock 140688667954624 on /run/cephadm/55ba4db6-0f0f-11ef-bc98-c7b262605968.lock 2024-05-10T20:54:24.373 INFO:teuthology.orchestra.run.smithi050.stdout:Lock 140688667954624 acquired on /run/cephadm/55ba4db6-0f0f-11ef-bc98-c7b262605968.lock 2024-05-10T20:54:24.374 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 3300 ... 2024-05-10T20:54:24.375 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 6789 ... 2024-05-10T20:54:24.376 INFO:teuthology.orchestra.run.smithi050.stdout:Base mon IP(s) is [172.21.15.50:3300, 172.21.15.50:6789], mon addrv is [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-05-10T20:54:24.391 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.50 metric 100 2024-05-10T20:54:24.391 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.50 metric 100 2024-05-10T20:54:24.404 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T20:54:24.404 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-10T20:54:24.404 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-10T20:54:24.416 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T20:54:24.417 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T20:54:24.417 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T20:54:24.417 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-10T20:54:24.417 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-05-10T20:54:24.417 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T20:54:24.419 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-05-10T20:54:24.419 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-05-10T20:54:24.420 INFO:teuthology.orchestra.run.smithi050.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T20:54:24.423 INFO:teuthology.orchestra.run.smithi050.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T20:54:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05... 2024-05-10T20:54:25.173 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05... 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:c63011eca3aa7db85b754e819bee7b4e168d1c0dc1623508a2394ec161d4e2fd 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying config sha256:d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 2024-05-10T20:54:25.175 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T20:54:26.473 INFO:teuthology.orchestra.run.smithi050.stdout:ceph: stdout ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev) 2024-05-10T20:54:26.473 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph version: ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev) 2024-05-10T20:54:26.473 INFO:teuthology.orchestra.run.smithi050.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T20:54:27.630 INFO:teuthology.orchestra.run.smithi050.stdout:stat: stdout 167 167 2024-05-10T20:54:27.630 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial keys... 2024-05-10T20:54:28.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQCEiT5mJSJpChAAan2nukBuDe5Dvu2bXXFRyA== 2024-05-10T20:54:29.953 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQCFiT5mqZo/FxAAw+/j/KYrek640gQ5bBEKIQ== 2024-05-10T20:54:31.224 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQCGiT5mw0ZeHxAA7JwySUzh6LAenIQ1vetEhw== 2024-05-10T20:54:31.227 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial monmap... 2024-05-10T20:54:32.418 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T20:54:32.418 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool for a [v2:172.21.15.50:3300,v1:172.21.15.50:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = quincy 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: set fsid to 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:54:32.419 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mon... 2024-05-10T20:54:33.616 INFO:teuthology.orchestra.run.smithi050.stdout:create mon.a on 2024-05-10T20:54:34.022 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-10T20:54:34.276 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T20:54:36.352 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target. 2024-05-10T20:54:36.352 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target. 2024-05-10T20:54:39.443 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a 2024-05-10T20:54:39.444 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T20:54:40.041 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target.wants/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@.service. 2024-05-10T20:54:40.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:40 smithi050 podman[29151]: 2024-05-10 20:54:40.388610106 +0000 UTC m=+0.035189421 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:54:44.365 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 podman[29151]: 2024-05-10 20:54:44.039842213 +0000 UTC m=+3.686421552 container create a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-10T20:54:44.365 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 podman[29151]: 2024-05-10 20:54:44.270356917 +0000 UTC m=+3.916936260 container init a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-10T20:54:44.365 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 podman[29151]: 2024-05-10 20:54:44.282007965 +0000 UTC m=+3.928587307 container start a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-10T20:54:44.426 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T20:54:44.426 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-10T20:54:44.444 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T20:54:44.445 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:54:44.445 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-10T20:54:44.445 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T20:54:44.445 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon to start... 2024-05-10T20:54:44.445 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon... 2024-05-10T20:54:44.641 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 bash[29151]: a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f 2024-05-10T20:54:44.641 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 systemd[1]: Started Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:54:44.641 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 ceph-mon[29169]: mkfs 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:44.641 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:44 smithi050 ceph-mon[29169]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T20:54:45.635 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: monmap epoch 1 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: last_changed 2024-05-10T20:54:31.773579+0000 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: min_mon_release 19 (squid) 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: election_strategy: 1 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: fsmap 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: mgrmap e1: no daemons active 2024-05-10T20:54:45.636 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:45 smithi050 ceph-mon[29169]: from='client.? 172.21.15.50:0/4164956729' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout id: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout services: 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.794271s) 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T20:54:45.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout data: 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.147+0000 7f443ba00700 1 Processor -- start 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.147+0000 7f443ba00700 1 -- start start 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.148+0000 7f443ba00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.148+0000 7f443ba00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f443410a5d0 con 0x7f4434109bd0 2024-05-10T20:54:45.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.149+0000 7f4439000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.149+0000 7f4439000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41950/0 (socket says 172.21.15.50:41950) 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.149+0000 7f4439000700 1 -- 172.21.15.50:0/3677500692 learned_addr learned my addr 172.21.15.50:0/3677500692 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.150+0000 7f4439000700 1 -- 172.21.15.50:0/3677500692 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443410ada0 con 0x7f4434109bd0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.150+0000 7f4439000700 1 --2- 172.21.15.50:0/3677500692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44280098a0 tx=0x7f4428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4033dd4f1abcc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.150+0000 7f4433400700 1 -- 172.21.15.50:0/3677500692 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428004160 con 0x7f4434109bd0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.150+0000 7f4433400700 1 -- 172.21.15.50:0/3677500692 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4428004300 con 0x7f4434109bd0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.150+0000 7f4433400700 1 -- 172.21.15.50:0/3677500692 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44280036a0 con 0x7f4434109bd0 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.151+0000 7f443ba00700 1 -- 172.21.15.50:0/3677500692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 msgr2=0x7f4434109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.151+0000 7f443ba00700 1 --2- 172.21.15.50:0/3677500692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44280098a0 tx=0x7f4428009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:45.929 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.151+0000 7f443ba00700 1 -- 172.21.15.50:0/3677500692 shutdown_connections 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.151+0000 7f443ba00700 1 --2- 172.21.15.50:0/3677500692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f4434109fe0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.151+0000 7f443ba00700 1 -- 172.21.15.50:0/3677500692 >> 172.21.15.50:0/3677500692 conn(0x7f4434105210 msgr2=0x7f44341076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.152+0000 7f443ba00700 1 -- 172.21.15.50:0/3677500692 shutdown_connections 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.152+0000 7f443ba00700 1 -- 172.21.15.50:0/3677500692 wait complete. 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.153+0000 7f443ba00700 1 Processor -- start 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.153+0000 7f443ba00700 1 -- start start 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.153+0000 7f443ba00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.153+0000 7f443ba00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f443410b2d0 con 0x7f4434109bd0 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.153+0000 7f4439000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:45.930 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.154+0000 7f4439000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41966/0 (socket says 172.21.15.50:41966) 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.154+0000 7f4439000700 1 -- 172.21.15.50:0/4164956729 learned_addr learned my addr 172.21.15.50:0/4164956729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.154+0000 7f4439000700 1 -- 172.21.15.50:0/4164956729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419fe90 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.154+0000 7f4439000700 1 --2- 172.21.15.50:0/4164956729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4428000c00 tx=0x7f4428004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428003db0 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4428011630 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341a0120 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4428011950 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a2d30 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f442801a460 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.155+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f442801b780 con 0x7f4434109bd0 2024-05-10T20:54:45.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.156+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434064e20 con 0x7f4434109bd0 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.160+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4428031020 con 0x7f4434109bd0 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.241+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f44341a3020 con 0x7f4434109bd0 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.242+0000 7f4431600700 1 -- 172.21.15.50:0/4164956729 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f442801ba20 con 0x7f4434109bd0 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.243+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 msgr2=0x7f443419f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.243+0000 7f443ba00700 1 --2- 172.21.15.50:0/4164956729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4428000c00 tx=0x7f4428004750 comp rx=0 tx=0).stop 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.244+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 shutdown_connections 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.244+0000 7f443ba00700 1 --2- 172.21.15.50:0/4164956729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4434109bd0 0x7f443419f950 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.244+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 >> 172.21.15.50:0/4164956729 conn(0x7f4434105210 msgr2=0x7f443407e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.244+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 shutdown_connections 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:45.244+0000 7f443ba00700 1 -- 172.21.15.50:0/4164956729 wait complete. 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:mon is available 2024-05-10T20:54:45.932 INFO:teuthology.orchestra.run.smithi050.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T20:54:47.117 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:46 smithi050 ceph-mon[29169]: from='client.? 172.21.15.50:0/2794091277' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T20:54:47.117 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:46 smithi050 ceph-mon[29169]: from='client.? 172.21.15.50:0/2794091277' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T20:54:47.430 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T20:54:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T20:54:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-05-10T20:54:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:47.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.682+0000 7f7183800700 1 Processor -- start 2024-05-10T20:54:47.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.682+0000 7f7183800700 1 -- start start 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.682+0000 7f7183800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.682+0000 7f7183800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10c800 con 0x7f717c10be00 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.683+0000 7f7180e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.683+0000 7f7180e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41980/0 (socket says 172.21.15.50:41980) 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.683+0000 7f7180e00700 1 -- 172.21.15.50:0/4272029387 learned_addr learned my addr 172.21.15.50:0/4272029387 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.683+0000 7f7180e00700 1 -- 172.21.15.50:0/4272029387 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c10cfd0 con 0x7f717c10be00 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.684+0000 7f7180e00700 1 --2- 172.21.15.50:0/4272029387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71700098a0 tx=0x7f7170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7d08466d63e982 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.684+0000 7f717b400700 1 -- 172.21.15.50:0/4272029387 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170004160 con 0x7f717c10be00 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.684+0000 7f717b400700 1 -- 172.21.15.50:0/4272029387 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7170004300 con 0x7f717c10be00 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.684+0000 7f717b400700 1 -- 172.21.15.50:0/4272029387 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71700036a0 con 0x7f717c10be00 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 -- 172.21.15.50:0/4272029387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 msgr2=0x7f717c10c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:47.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 --2- 172.21.15.50:0/4272029387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71700098a0 tx=0x7f7170009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 -- 172.21.15.50:0/4272029387 shutdown_connections 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 --2- 172.21.15.50:0/4272029387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c10c210 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 -- 172.21.15.50:0/4272029387 >> 172.21.15.50:0/4272029387 conn(0x7f717c07f3b0 msgr2=0x7f717c109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 -- 172.21.15.50:0/4272029387 shutdown_connections 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.685+0000 7f7183800700 1 -- 172.21.15.50:0/4272029387 wait complete. 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.686+0000 7f7183800700 1 Processor -- start 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7183800700 1 -- start start 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7183800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7183800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10d500 con 0x7f717c10be00 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7180e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7180e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41988/0 (socket says 172.21.15.50:41988) 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.687+0000 7f7180e00700 1 -- 172.21.15.50:0/2794091277 learned_addr learned my addr 172.21.15.50:0/2794091277 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:47.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.688+0000 7f7180e00700 1 -- 172.21.15.50:0/2794091277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c19fee0 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.688+0000 7f7180e00700 1 --2- 172.21.15.50:0/2794091277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7170000c00 tx=0x7f7170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.688+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170003db0 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.688+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7170011630 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.689+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a0170 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.689+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170011950 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.689+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a2d80 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.689+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f717001a460 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.689+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f717001b780 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.690+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c064e20 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.693+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f717001b980 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.771+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f717c02e7e0 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.783+0000 7f7179600700 1 -- 172.21.15.50:0/2794091277 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f717001bb60 con 0x7f717c10be00 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.784+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 msgr2=0x7f717c19f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:47.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.784+0000 7f7183800700 1 --2- 172.21.15.50:0/2794091277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7170000c00 tx=0x7f7170004750 comp rx=0 tx=0).stop 2024-05-10T20:54:47.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.784+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 shutdown_connections 2024-05-10T20:54:47.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.784+0000 7f7183800700 1 --2- 172.21.15.50:0/2794091277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f717c10be00 0x7f717c19f9a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:47.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.784+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 >> 172.21.15.50:0/2794091277 conn(0x7f717c07f3b0 msgr2=0x7f717c196000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:47.443 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.785+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 shutdown_connections 2024-05-10T20:54:47.443 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:46.785+0000 7f7183800700 1 -- 172.21.15.50:0/2794091277 wait complete. 2024-05-10T20:54:47.443 INFO:teuthology.orchestra.run.smithi050.stdout:Generating new minimal ceph.conf... 2024-05-10T20:54:48.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:48 smithi050 ceph-mon[29169]: from='client.? 172.21.15.50:0/3645930726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:54:48.875 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.106+0000 7f8880600700 1 Processor -- start 2024-05-10T20:54:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.107+0000 7f8880600700 1 -- start start 2024-05-10T20:54:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.107+0000 7f8880600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:48.876 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.107+0000 7f8880600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8878112210 con 0x7f8878111830 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.108+0000 7f887dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.108+0000 7f887dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41996/0 (socket says 172.21.15.50:41996) 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.108+0000 7f887dc00700 1 -- 172.21.15.50:0/1712929907 learned_addr learned my addr 172.21.15.50:0/1712929907 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.108+0000 7f887dc00700 1 -- 172.21.15.50:0/1712929907 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8878112390 con 0x7f8878111830 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.109+0000 7f887dc00700 1 --2- 172.21.15.50:0/1712929907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f886c0098a0 tx=0x7f886c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f620ec39fe7a709 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:48.877 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.109+0000 7f8877e00700 1 -- 172.21.15.50:0/1712929907 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c004160 con 0x7f8878111830 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.109+0000 7f8877e00700 1 -- 172.21.15.50:0/1712929907 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f886c004300 con 0x7f8878111830 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.109+0000 7f8877e00700 1 -- 172.21.15.50:0/1712929907 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c011af0 con 0x7f8878111830 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.110+0000 7f8880600700 1 -- 172.21.15.50:0/1712929907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 msgr2=0x7f8878111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.110+0000 7f8880600700 1 --2- 172.21.15.50:0/1712929907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f886c0098a0 tx=0x7f886c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.111+0000 7f8880600700 1 -- 172.21.15.50:0/1712929907 shutdown_connections 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.111+0000 7f8880600700 1 --2- 172.21.15.50:0/1712929907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f8878111c40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.111+0000 7f8880600700 1 -- 172.21.15.50:0/1712929907 >> 172.21.15.50:0/1712929907 conn(0x7f887807f3b0 msgr2=0x7f8878081800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.111+0000 7f8880600700 1 -- 172.21.15.50:0/1712929907 shutdown_connections 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.111+0000 7f8880600700 1 -- 172.21.15.50:0/1712929907 wait complete. 2024-05-10T20:54:48.879 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.112+0000 7f8880600700 1 Processor -- start 2024-05-10T20:54:48.880 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.112+0000 7f8880600700 1 -- start start 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.112+0000 7f8880600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.112+0000 7f8880600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8878114790 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.113+0000 7f887dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.113+0000 7f887dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42004/0 (socket says 172.21.15.50:42004) 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.113+0000 7f887dc00700 1 -- 172.21.15.50:0/3645930726 learned_addr learned my addr 172.21.15.50:0/3645930726 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.113+0000 7f887dc00700 1 -- 172.21.15.50:0/3645930726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781a4be0 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.113+0000 7f887dc00700 1 --2- 172.21.15.50:0/3645930726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f886c0090e0 tx=0x7f886c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c0039e0 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f886c003b80 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c0246e0 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88781a4e70 con 0x7f8878111830 2024-05-10T20:54:48.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a7a80 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f886c022020 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.114+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f886c02dcd0 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.115+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8878064e20 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.118+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8878112390 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.196+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f88781a7c50 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.197+0000 7f8876000700 1 -- 172.21.15.50:0/3645930726 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f886c01aae0 con 0x7f8878111830 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.198+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 msgr2=0x7f88781a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.198+0000 7f8880600700 1 --2- 172.21.15.50:0/3645930726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f886c0090e0 tx=0x7f886c011940 comp rx=0 tx=0).stop 2024-05-10T20:54:48.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.198+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 shutdown_connections 2024-05-10T20:54:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.198+0000 7f8880600700 1 --2- 172.21.15.50:0/3645930726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8878111830 0x7f88781a46a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.198+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 >> 172.21.15.50:0/3645930726 conn(0x7f887807f3b0 msgr2=0x7f8878080d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.199+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 shutdown_connections 2024-05-10T20:54:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:48.199+0000 7f8880600700 1 -- 172.21.15.50:0/3645930726 wait complete. 2024-05-10T20:54:48.885 INFO:teuthology.orchestra.run.smithi050.stdout:Restarting the monitor... 2024-05-10T20:54:49.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:48 smithi050 systemd[1]: Stopping Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T20:54:49.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29165]: 2024-05-10T20:54:49.572+0000 7f90d0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T20:54:49.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29165]: 2024-05-10T20:54:49.572+0000 7f90d0600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T20:54:49.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:49 smithi050 podman[29421]: 2024-05-10 20:54:49.697027721 +0000 UTC m=+0.460240865 container died a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T20:54:50.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 podman[29421]: 2024-05-10 20:54:50.065350204 +0000 UTC m=+0.828563338 container cleanup a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=) 2024-05-10T20:54:50.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 bash[29421]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a 2024-05-10T20:54:50.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 podman[29435]: 2024-05-10 20:54:50.21595294 +0000 UTC m=+0.506709931 container remove a16ea97fdf331a3eb7372c2c28f031f335a7081ba1c3126c0b13acc914e0227f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-10T20:54:51.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 systemd[1]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-10T20:54:51.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 systemd[1]: Stopped Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:54:51.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 systemd[1]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service: Consumed 2.122s CPU time. 2024-05-10T20:54:51.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:50 smithi050 systemd[1]: Starting Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T20:54:51.392 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 podman[29527]: 2024-05-10 20:54:51.024048624 +0000 UTC m=+0.034020635 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:54:51.392 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 podman[29527]: 2024-05-10 20:54:51.15330989 +0000 UTC m=+0.163281925 container create 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-10T20:54:51.392 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 podman[29527]: 2024-05-10 20:54:51.387568148 +0000 UTC m=+0.397540190 container init 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-05-10T20:54:51.524 INFO:teuthology.orchestra.run.smithi050.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 podman[29527]: 2024-05-10 20:54:51.399360202 +0000 UTC m=+0.409332221 container start 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 bash[29527]: 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 systemd[1]: Started Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev), process ceph-mon, pid 2 2024-05-10T20:54:51.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: pidfile_write: ignore empty --pid-file 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: load: jerasure load: lrc 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: RocksDB version: 7.9.2 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Git sha 0 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Compile date 2024-05-09 13:50:00 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: DB SUMMARY 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: DB Session ID: HAEN8LBZ2LWNNFL1LCQK 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: CURRENT file: CURRENT 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104978 ; 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.error_if_exists: 0 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.create_if_missing: 0 2024-05-10T20:54:51.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.paranoid_checks: 1 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.env: 0x55a52caff500 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.info_log: 0x55a5494c6060 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.statistics: (nil) 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.use_fsync: 0 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_log_file_size: 0 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T20:54:51.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_fallocate: 1 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.use_direct_reads: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.db_log_dir: 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.wal_dir: 2024-05-10T20:54:51.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.write_buffer_manager: 0x55a5493908c0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T20:54:51.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.unordered_write: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.row_cache: None 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.wal_filter: None 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.two_write_queues: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T20:54:51.679 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.wal_compression: 0 2024-05-10T20:54:51.680 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.atomic_flush: 0 2024-05-10T20:54:51.680 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T20:54:51.680 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.log_readahead_size: 0 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_background_jobs: 2 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_background_compactions: -1 2024-05-10T20:54:51.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_subcompactions: 1 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_open_files: -1 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T20:54:51.682 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_background_flushes: -1 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Compression algorithms supported: 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kZSTD supported: 0 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kXpressCompression supported: 0 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kZlibCompression supported: 1 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kSnappyCompression supported: 1 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kLZ4Compression supported: 1 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: kBZip2Compression supported: 0 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T20:54:51.683 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.merge_operator: 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_filter: None 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5494c6220) 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-05-10T20:54:51.684 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_type: 0 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_index_type: 0 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_shortening: 1 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: checksum: 4 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: no_block_cache: 0 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache: 0x55a5493c31f0 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_options: 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: capacity : 536870912 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_shard_bits : 4 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: strict_capacity_limit : 0 2024-05-10T20:54:51.685 INFO:journalctl@ceph.mon.a.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_compressed: (nil) 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: persistent_cache: (nil) 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size: 4096 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size_deviation: 10 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_restart_interval: 16 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_block_restart_interval: 1 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: metadata_block_size: 4096 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: partition_filters: 0 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: use_delta_encoding: 1 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: filter_policy: bloomfilter 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: whole_key_filtering: 1 2024-05-10T20:54:51.686 INFO:journalctl@ceph.mon.a.smithi050.stdout: verify_compression: 0 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: format_version: 5 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_index_compression: 1 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_align: 0 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: max_auto_readahead_size: 262144 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: prepopulate_block_cache: 0 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: initial_auto_readahead_size: 8192 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression: NoCompression 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T20:54:51.687 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.num_levels: 7 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T20:54:51.688 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T20:54:51.689 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T20:54:51.690 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T20:54:51.691 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.inplace_update_support: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.bloom_locality: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.max_successive_merges: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.ttl: 2592000 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T20:54:51.692 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enable_blob_files: false 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.min_blob_size: 0 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T20:54:51.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0d130ab7-5a94-4b7f-abc9-fa0fb50ac3a0 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374491534942, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T20:54:51.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374491537961, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98031, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14153, "raw_average_key_size": 49, "raw_value_size": 90330, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715374491, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0d130ab7-5a94-4b7f-abc9-fa0fb50ac3a0", "db_session_id": "HAEN8LBZ2LWNNFL1LCQK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374491538420, "job": 1, "event": "recovery_finished"} 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a5493a7c00 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: DB pointer 0x55a54e19a000 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** DB Stats ** 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:54:51.695 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: L0 2/0 99.66 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Sum 2/0 99.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 58.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-10T20:54:51.696 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative compaction: 0.00 GB write, 8.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval compaction: 0.00 GB write, 8.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache BinnedLRUCache@0x55a5493c31f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-10T20:54:51.697 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: starting mon.a rank 0 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???) e1 preinit fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).mds e1 new map 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).mds e1 print_map 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: e1 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: legacy client fscid: -1 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout: No filesystems configured 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T20:54:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: monmap epoch 1 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: last_changed 2024-05-10T20:54:31.773579+0000 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: min_mon_release 19 (squid) 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: election_strategy: 1 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: fsmap 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T20:54:51.699 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:51 smithi050 ceph-mon[29546]: mgrmap e1: no daemons active 2024-05-10T20:54:53.045 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.332+0000 7f3503e00700 1 Processor -- start 2024-05-10T20:54:53.046 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.333+0000 7f3503e00700 1 -- start start 2024-05-10T20:54:53.046 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.333+0000 7f3503e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:53.046 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.333+0000 7f3503e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f350410c650 con 0x7f350410bc50 2024-05-10T20:54:53.046 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.334+0000 7f3502a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:53.046 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.334+0000 7f3502a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42010/0 (socket says 172.21.15.50:42010) 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.334+0000 7f3502a00700 1 -- 172.21.15.50:0/1348315328 learned_addr learned my addr 172.21.15.50:0/1348315328 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.335+0000 7f3502a00700 1 -- 172.21.15.50:0/1348315328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350410ce20 con 0x7f350410bc50 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.335+0000 7f3502a00700 1 --2- 172.21.15.50:0/1348315328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098a0 tx=0x7f34f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e243be5111fc30a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.335+0000 7f3501600700 1 -- 172.21.15.50:0/1348315328 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4004160 con 0x7f350410bc50 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.335+0000 7f3501600700 1 -- 172.21.15.50:0/1348315328 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f34f4004300 con 0x7f350410bc50 2024-05-10T20:54:53.047 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.335+0000 7f3501600700 1 -- 172.21.15.50:0/1348315328 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4011af0 con 0x7f350410bc50 2024-05-10T20:54:53.048 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.336+0000 7f3503e00700 1 -- 172.21.15.50:0/1348315328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 msgr2=0x7f350410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.336+0000 7f3503e00700 1 --2- 172.21.15.50:0/1348315328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098a0 tx=0x7f34f4009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.336+0000 7f3503e00700 1 -- 172.21.15.50:0/1348315328 shutdown_connections 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.336+0000 7f3503e00700 1 --2- 172.21.15.50:0/1348315328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f350410c060 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.336+0000 7f3503e00700 1 -- 172.21.15.50:0/1348315328 >> 172.21.15.50:0/1348315328 conn(0x7f35041074b0 msgr2=0x7f35041098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.338+0000 7f3503e00700 1 -- 172.21.15.50:0/1348315328 shutdown_connections 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.338+0000 7f3503e00700 1 -- 172.21.15.50:0/1348315328 wait complete. 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.338+0000 7f3503e00700 1 Processor -- start 2024-05-10T20:54:53.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.338+0000 7f3503e00700 1 -- start start 2024-05-10T20:54:53.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3503e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:53.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3503e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35041101d0 con 0x7f350410bc50 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3502a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3502a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42024/0 (socket says 172.21.15.50:42024) 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3502a00700 1 -- 172.21.15.50:0/71927843 learned_addr learned my addr 172.21.15.50:0/71927843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.339+0000 7f3502a00700 1 -- 172.21.15.50:0/71927843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35041a0630 con 0x7f350410bc50 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f3502a00700 1 --2- 172.21.15.50:0/71927843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f34f40090e0 tx=0x7f34f4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f40039e0 con 0x7f350410bc50 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041a08c0 con 0x7f350410bc50 2024-05-10T20:54:53.051 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f34f4003b80 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34f4024690 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.340+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a34d0 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.341+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34f4022020 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.341+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f34f402dc40 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.342+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3504064e20 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.345+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f34f401a9c0 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.423+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f35041a36a0 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.428+0000 7f34fb400700 1 -- 172.21.15.50:0/71927843 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f34f402d3f0 con 0x7f350410bc50 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 msgr2=0x7f35041a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:53.053 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 --2- 172.21.15.50:0/71927843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f34f40090e0 tx=0x7f34f4011940 comp rx=0 tx=0).stop 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 shutdown_connections 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 --2- 172.21.15.50:0/71927843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f350410bc50 0x7f35041a00f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 >> 172.21.15.50:0/71927843 conn(0x7f35041074b0 msgr2=0x7f3504108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 shutdown_connections 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:52.430+0000 7f3503e00700 1 -- 172.21.15.50:0/71927843 wait complete. 2024-05-10T20:54:53.054 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T20:54:53.079 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T20:54:53.079 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mgr... 2024-05-10T20:54:53.081 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T20:54:53.082 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T20:54:53.542 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x 2024-05-10T20:54:53.542 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T20:54:53.560 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:53 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/71927843' entity='client.admin' 2024-05-10T20:54:53.818 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968.target.wants/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@.service. 2024-05-10T20:54:54.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:53 smithi050 systemd[1]: Starting Ceph mgr.x for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T20:54:54.521 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 podman[29763]: 2024-05-10 20:54:54.131214536 +0000 UTC m=+0.034008006 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T20:54:54.521 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 podman[29763]: 2024-05-10 20:54:54.269723217 +0000 UTC m=+0.172516693 container create 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T20:54:54.521 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 podman[29763]: 2024-05-10 20:54:54.515485872 +0000 UTC m=+0.418279383 container init 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T20:54:54.637 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T20:54:54.638 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-10T20:54:54.657 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T20:54:54.657 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:54:54.657 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-10T20:54:54.658 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T20:54:54.676 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T20:54:54.676 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-10T20:54:54.691 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T20:54:54.691 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:54:54.692 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-10T20:54:54.692 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T20:54:54.692 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr to start... 2024-05-10T20:54:54.692 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr... 2024-05-10T20:54:54.829 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 podman[29763]: 2024-05-10 20:54:54.526523041 +0000 UTC m=+0.429316520 container start 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-10T20:54:54.829 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 bash[29763]: 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 2024-05-10T20:54:54.829 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 systemd[1]: Started Ceph mgr.x for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:54:55.114 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:54.855+0000 7effa1573200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T20:54:55.422 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:55.131+0000 7effa1573200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:54:55.422 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:55.273+0000 7effa1573200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T20:54:55.715 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:55 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/523566865' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T20:54:56.149 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:55.709+0000 7effa1573200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T20:54:56.149 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:55.831+0000 7effa1573200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T20:54:56.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:56.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T20:54:56.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T20:54:56.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:56.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T20:54:56.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T20:54:56.177 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T20:54:56.177 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T20:54:56.177 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T20:54:56.178 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T20:54:44.452412+0000", 2024-05-10T20:54:56.179 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.341+0000 7f6096000700 1 Processor -- start 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.341+0000 7f6096000700 1 -- start start 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.341+0000 7f6096000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.341+0000 7f6096000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f609010a620 con 0x7f6090109c20 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.342+0000 7f608f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.342+0000 7f608f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45028/0 (socket says 172.21.15.50:45028) 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.342+0000 7f608f400700 1 -- 172.21.15.50:0/3718763807 learned_addr learned my addr 172.21.15.50:0/3718763807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.342+0000 7f608f400700 1 -- 172.21.15.50:0/3718763807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609010adf0 con 0x7f6090109c20 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.343+0000 7f608f400700 1 --2- 172.21.15.50:0/3718763807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60780098a0 tx=0x7f6078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d251b5daa7e27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:56.180 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.343+0000 7f608e000700 1 -- 172.21.15.50:0/3718763807 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6078004160 con 0x7f6090109c20 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.343+0000 7f608e000700 1 -- 172.21.15.50:0/3718763807 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6078004300 con 0x7f6090109c20 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.343+0000 7f608e000700 1 -- 172.21.15.50:0/3718763807 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6078011af0 con 0x7f6090109c20 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 -- 172.21.15.50:0/3718763807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 msgr2=0x7f609010a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 --2- 172.21.15.50:0/3718763807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60780098a0 tx=0x7f6078009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 -- 172.21.15.50:0/3718763807 shutdown_connections 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 --2- 172.21.15.50:0/3718763807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609010a030 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 -- 172.21.15.50:0/3718763807 >> 172.21.15.50:0/3718763807 conn(0x7f60901052a0 msgr2=0x7f6090107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 -- 172.21.15.50:0/3718763807 shutdown_connections 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.344+0000 7f6096000700 1 -- 172.21.15.50:0/3718763807 wait complete. 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.345+0000 7f6096000700 1 Processor -- start 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f6096000700 1 -- start start 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f6096000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f6096000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f609010e020 con 0x7f6090109c20 2024-05-10T20:54:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f608f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:56.182 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f608f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45030/0 (socket says 172.21.15.50:45030) 2024-05-10T20:54:56.182 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.346+0000 7f608f400700 1 -- 172.21.15.50:0/523566865 learned_addr learned my addr 172.21.15.50:0/523566865 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f608f400700 1 -- 172.21.15.50:0/523566865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609019e5d0 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f608f400700 1 --2- 172.21.15.50:0/523566865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60780090e0 tx=0x7f6078011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60780039e0 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6078003b80 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6078024690 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.347+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609019e860 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.348+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60901a1470 con 0x7f6090109c20 2024-05-10T20:54:56.184 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.349+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6090064e20 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.349+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6078022020 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.349+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6078024bd0 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.351+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6078029060 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.432+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f60901a1760 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.432+0000 7f6087e00700 1 -- 172.21.15.50:0/523566865 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6078024e20 con 0x7f6090109c20 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 msgr2=0x7f609019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 --2- 172.21.15.50:0/523566865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60780090e0 tx=0x7f6078011940 comp rx=0 tx=0).stop 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 shutdown_connections 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 --2- 172.21.15.50:0/523566865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6090109c20 0x7f609019e090 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 >> 172.21.15.50:0/523566865 conn(0x7f60901052a0 msgr2=0x7f6090105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 shutdown_connections 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:55.434+0000 7f6096000700 1 -- 172.21.15.50:0/523566865 wait complete. 2024-05-10T20:54:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (1/15)... 2024-05-10T20:54:57.834 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:57.499+0000 7effa1573200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T20:54:57.834 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:57.738+0000 7effa1573200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T20:54:58.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:57.967+0000 7effa1573200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T20:54:58.621 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:58.380+0000 7effa1573200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T20:54:58.621 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:58.517+0000 7effa1573200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T20:54:58.922 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:58.784+0000 7effa1573200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T20:54:59.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:54:58 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3782782128' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T20:54:59.339 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:59.055+0000 7effa1573200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:54:59.660 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:59.661 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T20:54:59.662 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T20:54:59.663 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T20:54:59.664 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T20:54:44.452412+0000", 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.846+0000 7fb40a800700 1 Processor -- start 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.846+0000 7fb40a800700 1 -- start start 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.847+0000 7fb40a800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.847+0000 7fb40a800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410a430 con 0x7fb404109a30 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.847+0000 7fb409400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.847+0000 7fb409400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45040/0 (socket says 172.21.15.50:45040) 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.847+0000 7fb409400700 1 -- 172.21.15.50:0/3644626364 learned_addr learned my addr 172.21.15.50:0/3644626364 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.848+0000 7fb409400700 1 -- 172.21.15.50:0/3644626364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40410ac00 con 0x7fb404109a30 2024-05-10T20:54:59.665 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.848+0000 7fb409400700 1 --2- 172.21.15.50:0/3644626364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80098a0 tx=0x7fb3f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72dafad544bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.848+0000 7fb403e00700 1 -- 172.21.15.50:0/3644626364 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f8004160 con 0x7fb404109a30 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.848+0000 7fb403e00700 1 -- 172.21.15.50:0/3644626364 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb3f8004300 con 0x7fb404109a30 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.848+0000 7fb403e00700 1 -- 172.21.15.50:0/3644626364 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f8011af0 con 0x7fb404109a30 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 -- 172.21.15.50:0/3644626364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 msgr2=0x7fb404109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 --2- 172.21.15.50:0/3644626364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80098a0 tx=0x7fb3f8009bb0 comp rx=0 tx=0).stop 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 -- 172.21.15.50:0/3644626364 shutdown_connections 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 --2- 172.21.15.50:0/3644626364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb404109e40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 -- 172.21.15.50:0/3644626364 >> 172.21.15.50:0/3644626364 conn(0x7fb4041050d0 msgr2=0x7fb404107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 -- 172.21.15.50:0/3644626364 shutdown_connections 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.849+0000 7fb40a800700 1 -- 172.21.15.50:0/3644626364 wait complete. 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.851+0000 7fb40a800700 1 Processor -- start 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.851+0000 7fb40a800700 1 -- start start 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb40a800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:54:59.666 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb40a800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410dfb0 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb409400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb409400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45054/0 (socket says 172.21.15.50:45054) 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb409400700 1 -- 172.21.15.50:0/3782782128 learned_addr learned my addr 172.21.15.50:0/3782782128 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.852+0000 7fb409400700 1 -- 172.21.15.50:0/3782782128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40419e460 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb409400700 1 --2- 172.21.15.50:0/3782782128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80090e0 tx=0x7fb3f8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f80039e0 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb3f8003b80 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f8024690 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40419e6f0 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a1300 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.853+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb3f8022020 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.854+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3f80193c0 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.854+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb404064e20 con 0x7fb404109a30 2024-05-10T20:54:59.667 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.857+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb3f801a9c0 con 0x7fb404109a30 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.938+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4041a14d0 con 0x7fb404109a30 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.939+0000 7fb402000700 1 -- 172.21.15.50:0/3782782128 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb3f801a430 con 0x7fb404109a30 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.940+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 msgr2=0x7fb40419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.940+0000 7fb40a800700 1 --2- 172.21.15.50:0/3782782128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80090e0 tx=0x7fb3f8011940 comp rx=0 tx=0).stop 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.940+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 shutdown_connections 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.940+0000 7fb40a800700 1 --2- 172.21.15.50:0/3782782128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb404109a30 0x7fb40419df20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.940+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 >> 172.21.15.50:0/3782782128 conn(0x7fb4041050d0 msgr2=0x7fb404105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.941+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 shutdown_connections 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:54:58.941+0000 7fb40a800700 1 -- 172.21.15.50:0/3782782128 wait complete. 2024-05-10T20:54:59.668 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (2/15)... 2024-05-10T20:55:00.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:54:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:54:59.893+0000 7effa1573200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T20:55:00.672 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:00.197+0000 7effa1573200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T20:55:01.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:00.785+0000 7effa1573200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T20:55:01.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:00.921+0000 7effa1573200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T20:55:01.814 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:01.420+0000 7effa1573200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T20:55:02.083 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:01.900+0000 7effa1573200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T20:55:02.083 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:02.035+0000 7effa1573200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T20:55:02.391 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:02.386+0000 7effa1573200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T20:55:02.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:02 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2680293123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T20:55:02.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:02.583+0000 7effa1573200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T20:55:03.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:02.723+0000 7effa1573200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T20:55:03.104 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:55:03.104 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:03.104 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T20:55:03.104 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T20:55:03.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T20:55:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T20:55:03.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T20:55:03.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T20:55:03.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T20:55:03.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T20:55:03.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T20:55:03.115 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T20:55:03.116 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T20:54:44.452412+0000", 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.307+0000 7f88eb200700 1 Processor -- start 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.308+0000 7f88eb200700 1 -- start start 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.308+0000 7f88eb200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.308+0000 7f88eb200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec07d770 con 0x7f88ec07ee00 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.309+0000 7f88e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.309+0000 7f88e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45068/0 (socket says 172.21.15.50:45068) 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.309+0000 7f88e9e00700 1 -- 172.21.15.50:0/1093925213 learned_addr learned my addr 172.21.15.50:0/1093925213 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.309+0000 7f88e9e00700 1 -- 172.21.15.50:0/1093925213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec07d8f0 con 0x7f88ec07ee00 2024-05-10T20:55:03.117 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.309+0000 7f88e9e00700 1 --2- 172.21.15.50:0/1093925213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098a0 tx=0x7f88dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf3bbb50d83a8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.310+0000 7f88e8a00700 1 -- 172.21.15.50:0/1093925213 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88dc004160 con 0x7f88ec07ee00 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.310+0000 7f88e8a00700 1 -- 172.21.15.50:0/1093925213 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88dc004300 con 0x7f88ec07ee00 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.310+0000 7f88e8a00700 1 -- 172.21.15.50:0/1093925213 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88dc011af0 con 0x7f88ec07ee00 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 -- 172.21.15.50:0/1093925213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 msgr2=0x7f88ec07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 --2- 172.21.15.50:0/1093925213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098a0 tx=0x7f88dc009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 -- 172.21.15.50:0/1093925213 shutdown_connections 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 --2- 172.21.15.50:0/1093925213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec07d230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 -- 172.21.15.50:0/1093925213 >> 172.21.15.50:0/1093925213 conn(0x7f88ec105080 msgr2=0x7f88ec107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 -- 172.21.15.50:0/1093925213 shutdown_connections 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.311+0000 7f88eb200700 1 -- 172.21.15.50:0/1093925213 wait complete. 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.312+0000 7f88eb200700 1 Processor -- start 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88eb200700 1 -- start start 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88eb200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:03.118 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88eb200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec110000 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45080/0 (socket says 172.21.15.50:45080) 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.313+0000 7f88e9e00700 1 -- 172.21.15.50:0/2680293123 learned_addr learned my addr 172.21.15.50:0/2680293123 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.314+0000 7f88e9e00700 1 -- 172.21.15.50:0/2680293123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1a05b0 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.314+0000 7f88e9e00700 1 --2- 172.21.15.50:0/2680293123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0117b0 tx=0x7f88dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.314+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88dc0039c0 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.314+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88dc003b60 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.315+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1a0840 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.315+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88dc0196a0 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.315+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a3450 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.315+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f88dc019cf0 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.315+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88dc024b40 con 0x7f88ec07ee00 2024-05-10T20:55:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.316+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec064e20 con 0x7f88ec07ee00 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.319+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f88dc0042a0 con 0x7f88ec07ee00 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.400+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88ec1a3620 con 0x7f88ec07ee00 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.401+0000 7f88e2a00700 1 -- 172.21.15.50:0/2680293123 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f88dc024d40 con 0x7f88ec07ee00 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.402+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 msgr2=0x7f88ec1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.402+0000 7f88eb200700 1 --2- 172.21.15.50:0/2680293123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0117b0 tx=0x7f88dc011b70 comp rx=0 tx=0).stop 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.403+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 shutdown_connections 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.403+0000 7f88eb200700 1 --2- 172.21.15.50:0/2680293123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec07ee00 0x7f88ec1a0070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.403+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 >> 172.21.15.50:0/2680293123 conn(0x7f88ec105080 msgr2=0x7f88ec105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.403+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 shutdown_connections 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:02.403+0000 7f88eb200700 1 -- 172.21.15.50:0/2680293123 wait complete. 2024-05-10T20:55:03.120 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (3/15)... 2024-05-10T20:55:03.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:03.147+0000 7effa1573200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T20:55:03.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:03.279+0000 7effa1573200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T20:55:04.388 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:04.080+0000 7effa1573200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T20:55:04.672 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:04.382+0000 7effa1573200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: Activating manager daemon x 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: mgrmap e2: x(active, starting, since 0.0134736s) 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:55:04.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T20:55:04.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: Manager daemon x is now available 2024-05-10T20:55:04.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T20:55:04.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T20:55:04.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:04 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' 2024-05-10T20:55:05.540 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:05 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' 2024-05-10T20:55:05.540 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:05 smithi050 ceph-mon[29546]: from='mgr.14100 172.21.15.50:0/809535466' entity='mgr.x' 2024-05-10T20:55:05.540 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:05 smithi050 ceph-mon[29546]: mgrmap e3: x(active, since 1.0181s) 2024-05-10T20:55:06.770 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:06 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2625252314' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T20:55:06.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T20:55:06.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.786 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T20:55:06.786 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:06.786 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T20:55:06.786 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T20:55:06.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T20:55:06.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T20:55:06.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T20:55:06.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T20:55:06.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T20:55:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T20:54:44.452412+0000", 2024-05-10T20:55:06.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.774+0000 7f0f23e00700 1 Processor -- start 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.775+0000 7f0f23e00700 1 -- start start 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.775+0000 7f0f23e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.775+0000 7f0f23e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2407ce70 con 0x7f0f2410a500 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.775+0000 7f0f22a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.776+0000 7f0f22a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38590/0 (socket says 172.21.15.50:38590) 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.776+0000 7f0f22a00700 1 -- 172.21.15.50:0/243366074 learned_addr learned my addr 172.21.15.50:0/243366074 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.776+0000 7f0f22a00700 1 -- 172.21.15.50:0/243366074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2410ce60 con 0x7f0f2410a500 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.776+0000 7f0f22a00700 1 --2- 172.21.15.50:0/243366074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d170fe635cdeb9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.777+0000 7f0f21600700 1 -- 172.21.15.50:0/243366074 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f14004160 con 0x7f0f2410a500 2024-05-10T20:55:06.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.777+0000 7f0f21600700 1 -- 172.21.15.50:0/243366074 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f14004300 con 0x7f0f2410a500 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.777+0000 7f0f21600700 1 -- 172.21.15.50:0/243366074 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f14011af0 con 0x7f0f2410a500 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 -- 172.21.15.50:0/243366074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 msgr2=0x7f0f2410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 --2- 172.21.15.50:0/243366074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f140098a0 tx=0x7f0f14009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 -- 172.21.15.50:0/243366074 shutdown_connections 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 --2- 172.21.15.50:0/243366074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2410c920 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 -- 172.21.15.50:0/243366074 >> 172.21.15.50:0/243366074 conn(0x7f0f24103f60 msgr2=0x7f0f24106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 -- 172.21.15.50:0/243366074 shutdown_connections 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.778+0000 7f0f23e00700 1 -- 172.21.15.50:0/243366074 wait complete. 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.779+0000 7f0f23e00700 1 Processor -- start 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.779+0000 7f0f23e00700 1 -- start start 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f23e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f23e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410de20 con 0x7f0f2410a500 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f22a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:06.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f22a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38602/0 (socket says 172.21.15.50:38602) 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f22a00700 1 -- 172.21.15.50:0/2625252314 learned_addr learned my addr 172.21.15.50:0/2625252314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.780+0000 7f0f22a00700 1 -- 172.21.15.50:0/2625252314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2419e400 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f22a00700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f140117b0 tx=0x7f0f14011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f140039c0 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f14003b60 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2419e690 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f140196a0 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.781+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241a12a0 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.782+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f0f14019960 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.782+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f24064e20 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.783+0000 7f0f1b400700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 0x7f0f08040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.783+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f14053ae0 con 0x7f0f2410a500 2024-05-10T20:55:06.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.783+0000 7f0f22000700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 0x7f0f08040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.784+0000 7f0f22000700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 0x7f0f08040810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006fd0 tx=0x7f0f10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:05.790+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f1402ce20 con 0x7f0f2410a500 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.105+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0f241a1920 con 0x7f0f2410a500 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.106+0000 7f0f1b400700 1 -- 172.21.15.50:0/2625252314 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0f14023390 con 0x7f0f2410a500 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 msgr2=0x7f0f08040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 0x7f0f08040810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006fd0 tx=0x7f0f10006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 msgr2=0x7f0f2419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f140117b0 tx=0x7f0f14011b70 comp rx=0 tx=0).stop 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 shutdown_connections 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f0f0803e300 0x7f0f08040810 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 --2- 172.21.15.50:0/2625252314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2410a500 0x7f0f2419dec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 >> 172.21.15.50:0/2625252314 conn(0x7f0f24103f60 msgr2=0x7f0f24104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 shutdown_connections 2024-05-10T20:55:06.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:06.109+0000 7f0f23e00700 1 -- 172.21.15.50:0/2625252314 wait complete. 2024-05-10T20:55:06.804 INFO:teuthology.orchestra.run.smithi050.stdout:mgr is available 2024-05-10T20:55:07.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:07 smithi050 ceph-mon[29546]: mgrmap e4: x(active, since 2s) 2024-05-10T20:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T20:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T20:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.539+0000 7f488be00700 1 Processor -- start 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.539+0000 7f488be00700 1 -- start start 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.540+0000 7f488be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.540+0000 7f488be00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10c610 con 0x7f488c10bc10 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.540+0000 7f488aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.540+0000 7f488aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38608/0 (socket says 172.21.15.50:38608) 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.540+0000 7f488aa00700 1 -- 172.21.15.50:0/2285567927 learned_addr learned my addr 172.21.15.50:0/2285567927 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.541+0000 7f488aa00700 1 -- 172.21.15.50:0/2285567927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c10cde0 con 0x7f488c10bc10 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.541+0000 7f488aa00700 1 --2- 172.21.15.50:0/2285567927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0435e35eced5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.541+0000 7f4889600700 1 -- 172.21.15.50:0/2285567927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c004160 con 0x7f488c10bc10 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.541+0000 7f4889600700 1 -- 172.21.15.50:0/2285567927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487c004300 con 0x7f488c10bc10 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.541+0000 7f4889600700 1 -- 172.21.15.50:0/2285567927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c011af0 con 0x7f488c10bc10 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 -- 172.21.15.50:0/2285567927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 msgr2=0x7f488c10c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 --2- 172.21.15.50:0/2285567927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 -- 172.21.15.50:0/2285567927 shutdown_connections 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 --2- 172.21.15.50:0/2285567927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c10c020 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 -- 172.21.15.50:0/2285567927 >> 172.21.15.50:0/2285567927 conn(0x7f488c07f0e0 msgr2=0x7f488c109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.542+0000 7f488be00700 1 -- 172.21.15.50:0/2285567927 shutdown_connections 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.543+0000 7f488be00700 1 -- 172.21.15.50:0/2285567927 wait complete. 2024-05-10T20:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.544+0000 7f488be00700 1 Processor -- start 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.544+0000 7f488be00700 1 -- start start 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.544+0000 7f488be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.544+0000 7f488be00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f488c110190 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.544+0000 7f488aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.545+0000 7f488aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38610/0 (socket says 172.21.15.50:38610) 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.545+0000 7f488aa00700 1 -- 172.21.15.50:0/3366847723 learned_addr learned my addr 172.21.15.50:0/3366847723 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.545+0000 7f488aa00700 1 -- 172.21.15.50:0/3366847723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c1a05f0 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.545+0000 7f488aa00700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f487c011890 tx=0x7f487c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.546+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c0039e0 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.546+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487c003b80 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.546+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c1a0880 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.546+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c0246e0 con 0x7f488c10bc10 2024-05-10T20:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.546+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c1a3490 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.547+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c064e20 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.547+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f487c022020 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.548+0000 7f4883400700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 0x7f4870040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.548+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f487c0533e0 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.548+0000 7f488a000700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 0x7f4870040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.549+0000 7f488a000700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 0x7f4870040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4878006fd0 tx=0x7f4878006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.554+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f487c00f2e0 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.786+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f488c1a4c90 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.790+0000 7f4883400700 1 -- 172.21.15.50:0/3366847723 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f487c053140 con 0x7f488c10bc10 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 msgr2=0x7f4870040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 0x7f4870040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4878006fd0 tx=0x7f4878006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 msgr2=0x7f488c1a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f487c011890 tx=0x7f487c011970 comp rx=0 tx=0).stop 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 shutdown_connections 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f487003e420 0x7f4870040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 --2- 172.21.15.50:0/3366847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f488c10bc10 0x7f488c1a00b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 >> 172.21.15.50:0/3366847723 conn(0x7f488c07f0e0 msgr2=0x7f488c1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.793+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 shutdown_connections 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:07.794+0000 7f488be00700 1 -- 172.21.15.50:0/3366847723 wait complete. 2024-05-10T20:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling cephadm module... 2024-05-10T20:55:08.819 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:08 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3366847723' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T20:55:09.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:09 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2927104508' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T20:55:09.923 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: ignoring --setuser ceph since I am not root 2024-05-10T20:55:09.923 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: ignoring --setgroup ceph since I am not root 2024-05-10T20:55:10.207 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:09.947+0000 7f7fe9961200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T20:55:10.241 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.203+0000 7fc094a00700 1 Processor -- start 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.204+0000 7fc094a00700 1 -- start start 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.204+0000 7fc094a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.204+0000 7fc094a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc09010a5d0 con 0x7fc090109bd0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.205+0000 7fc08de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.205+0000 7fc08de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38618/0 (socket says 172.21.15.50:38618) 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.205+0000 7fc08de00700 1 -- 172.21.15.50:0/242085133 learned_addr learned my addr 172.21.15.50:0/242085133 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.205+0000 7fc08de00700 1 -- 172.21.15.50:0/242085133 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09010ada0 con 0x7fc090109bd0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.205+0000 7fc08de00700 1 --2- 172.21.15.50:0/242085133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a7e7a4a413e2f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.206+0000 7fc08ca00700 1 -- 172.21.15.50:0/242085133 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080004160 con 0x7fc090109bd0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.206+0000 7fc08ca00700 1 -- 172.21.15.50:0/242085133 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc080004300 con 0x7fc090109bd0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.206+0000 7fc08ca00700 1 -- 172.21.15.50:0/242085133 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080011af0 con 0x7fc090109bd0 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 -- 172.21.15.50:0/242085133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 msgr2=0x7fc090109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:10.242 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 --2- 172.21.15.50:0/242085133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 -- 172.21.15.50:0/242085133 shutdown_connections 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 --2- 172.21.15.50:0/242085133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc090109fe0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 -- 172.21.15.50:0/242085133 >> 172.21.15.50:0/242085133 conn(0x7fc090105210 msgr2=0x7fc0901076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 -- 172.21.15.50:0/242085133 shutdown_connections 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.207+0000 7fc094a00700 1 -- 172.21.15.50:0/242085133 wait complete. 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.208+0000 7fc094a00700 1 Processor -- start 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.208+0000 7fc094a00700 1 -- start start 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc094a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc094a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc09010e150 con 0x7fc090109bd0 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc08de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc08de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38622/0 (socket says 172.21.15.50:38622) 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc08de00700 1 -- 172.21.15.50:0/2927104508 learned_addr learned my addr 172.21.15.50:0/2927104508 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.209+0000 7fc08de00700 1 -- 172.21.15.50:0/2927104508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09019e5d0 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc08de00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc0800090e0 tx=0x7fc080011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0800039e0 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc080003b80 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09019e860 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080024690 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.210+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0901a1470 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.211+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc090064e20 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.212+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fc080022020 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.212+0000 7fc086a00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 0x7fc074040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.212+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc080053390 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.213+0000 7fc08d400700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 0x7fc074040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.213+0000 7fc08d400700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 0x7fc074040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006fd0 tx=0x7fc07c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.218+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc080016260 con 0x7fc090109bd0 2024-05-10T20:55:10.244 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.518+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc0901a1af0 con 0x7fc090109bd0 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.572+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc0800530f0 con 0x7fc090109bd0 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.576+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 msgr2=0x7fc074040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.576+0000 7fc094a00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 0x7fc074040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006fd0 tx=0x7fc07c006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.576+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 msgr2=0x7fc09019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.577+0000 7fc094a00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc0800090e0 tx=0x7fc080011940 comp rx=0 tx=0).stop 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.577+0000 7fc086a00700 1 -- 172.21.15.50:0/2927104508 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc08002ca40 con 0x7fc090109bd0 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.578+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 shutdown_connections 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.578+0000 7fc094a00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7fc07403e470 0x7fc074040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.578+0000 7fc094a00700 1 --2- 172.21.15.50:0/2927104508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc090109bd0 0x7fc09019e090 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.578+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 >> 172.21.15.50:0/2927104508 conn(0x7fc090105210 msgr2=0x7fc090105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:10.245 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.579+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 shutdown_connections 2024-05-10T20:55:10.246 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:09.579+0000 7fc094a00700 1 -- 172.21.15.50:0/2927104508 wait complete. 2024-05-10T20:55:10.534 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:10.202+0000 7f7fe9961200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:55:10.534 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:10.353+0000 7f7fe9961200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T20:55:10.802 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:10.776+0000 7f7fe9961200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T20:55:10.802 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:10 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2927104508' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T20:55:10.802 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:10 smithi050 ceph-mon[29546]: mgrmap e5: x(active, since 5s) 2024-05-10T20:55:11.150 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:10.899+0000 7f7fe9961200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T20:55:11.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:11 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3056241740' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T20:55:11.956 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.934+0000 7f7fcc200700 1 Processor -- start 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.935+0000 7f7fcc200700 1 -- start start 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.935+0000 7f7fcc200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.935+0000 7f7fcc200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc410a620 con 0x7f7fc4109c20 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.936+0000 7f7fc9800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.936+0000 7f7fc9800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38654/0 (socket says 172.21.15.50:38654) 2024-05-10T20:55:11.957 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.936+0000 7f7fc9800700 1 -- 172.21.15.50:0/1929859539 learned_addr learned my addr 172.21.15.50:0/1929859539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.936+0000 7f7fc9800700 1 -- 172.21.15.50:0/1929859539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc410adf0 con 0x7f7fc4109c20 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.937+0000 7f7fc9800700 1 --2- 172.21.15.50:0/1929859539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80098a0 tx=0x7f7fb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eea8c4375b1784 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.937+0000 7f7fc3e00700 1 -- 172.21.15.50:0/1929859539 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8004160 con 0x7f7fc4109c20 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.937+0000 7f7fc3e00700 1 -- 172.21.15.50:0/1929859539 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7fb8004300 con 0x7f7fc4109c20 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.937+0000 7f7fc3e00700 1 -- 172.21.15.50:0/1929859539 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8011af0 con 0x7f7fc4109c20 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 -- 172.21.15.50:0/1929859539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 msgr2=0x7f7fc410a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 --2- 172.21.15.50:0/1929859539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80098a0 tx=0x7f7fb8009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:11.958 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 -- 172.21.15.50:0/1929859539 shutdown_connections 2024-05-10T20:55:11.959 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 --2- 172.21.15.50:0/1929859539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc410a030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:11.959 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 -- 172.21.15.50:0/1929859539 >> 172.21.15.50:0/1929859539 conn(0x7f7fc41052a0 msgr2=0x7f7fc4107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 -- 172.21.15.50:0/1929859539 shutdown_connections 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.938+0000 7f7fcc200700 1 -- 172.21.15.50:0/1929859539 wait complete. 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fcc200700 1 Processor -- start 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fcc200700 1 -- start start 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fcc200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fcc200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc410e020 con 0x7f7fc4109c20 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fc9800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fc9800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38656/0 (socket says 172.21.15.50:38656) 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.940+0000 7f7fc9800700 1 -- 172.21.15.50:0/3056241740 learned_addr learned my addr 172.21.15.50:0/3056241740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.941+0000 7f7fc9800700 1 -- 172.21.15.50:0/3056241740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc41a07e0 con 0x7f7fc4109c20 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.941+0000 7f7fc9800700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80090e0 tx=0x7f7fb8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.942+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb80039e0 con 0x7f7fc4109c20 2024-05-10T20:55:11.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.942+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7fb8003b80 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.942+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8024690 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.942+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc41a0a70 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.942+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a3680 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.943+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4064e20 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.944+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7fb8022020 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.944+0000 7f7fc2000700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 0x7f7fac0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.944+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fb80533f0 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.944+0000 7f7fc8e00700 1 -- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 msgr2=0x7f7fac0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.944+0000 7f7fc8e00700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 0x7f7fac0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:10.954+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fb8053150 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.145+0000 7f7fc8e00700 1 -- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 msgr2=0x7f7fac0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.145+0000 7f7fc8e00700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 0x7f7fac0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.251+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7fc41a3850 con 0x7f7fc4109c20 2024-05-10T20:55:11.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.252+0000 7f7fc2000700 1 -- 172.21.15.50:0/3056241740 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f7fc41a3850 con 0x7f7fc4109c20 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 msgr2=0x7f7fac0409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 0x7f7fac0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 msgr2=0x7f7fc41a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80090e0 tx=0x7f7fb8011940 comp rx=0 tx=0).stop 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 shutdown_connections 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f7fac03e4c0 0x7f7fac0409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 --2- 172.21.15.50:0/3056241740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fc4109c20 0x7f7fc41a02a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 >> 172.21.15.50:0/3056241740 conn(0x7f7fc41052a0 msgr2=0x7f7fc4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 shutdown_connections 2024-05-10T20:55:11.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:11.255+0000 7f7fcc200700 1 -- 172.21.15.50:0/3056241740 wait complete. 2024-05-10T20:55:11.966 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-05-10T20:55:11.966 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 5... 2024-05-10T20:55:12.789 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:12.538+0000 7f7fe9961200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T20:55:13.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:12.783+0000 7f7fe9961200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T20:55:13.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:13.006+0000 7f7fe9961200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T20:55:13.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:13.404+0000 7f7fe9961200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T20:55:13.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:13.534+0000 7f7fe9961200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T20:55:14.068 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:13.795+0000 7f7fe9961200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T20:55:14.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:14.063+0000 7f7fe9961200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:55:15.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:14.910+0000 7f7fe9961200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T20:55:15.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:15.218+0000 7f7fe9961200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T20:55:16.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:15.819+0000 7f7fe9961200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T20:55:16.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:15.957+0000 7f7fe9961200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T20:55:16.710 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:16.442+0000 7f7fe9961200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T20:55:17.054 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:16.919+0000 7f7fe9961200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T20:55:17.394 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:17.049+0000 7f7fe9961200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T20:55:17.394 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:17.389+0000 7f7fe9961200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T20:55:17.672 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:17.580+0000 7f7fe9961200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T20:55:18.136 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:17.714+0000 7f7fe9961200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T20:55:18.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:18.130+0000 7f7fe9961200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T20:55:18.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:18.259+0000 7f7fe9961200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T20:55:19.361 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:19.056+0000 7f7fe9961200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T20:55:19.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:19.355+0000 7f7fe9961200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T20:55:19.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: Active manager daemon x restarted 2024-05-10T20:55:19.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: Activating manager daemon x 2024-05-10T20:55:19.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T20:55:19.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: mgrmap e6: x(active, starting, since 0.0100243s) 2024-05-10T20:55:19.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:55:19.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T20:55:19.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T20:55:19.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T20:55:19.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T20:55:19.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:19 smithi050 ceph-mon[29546]: Manager daemon x is now available 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: Found migration_current of "None". Setting to last migration. 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:20.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:20 smithi050 ceph-mon[29546]: mgrmap e7: x(active, since 1.01312s) 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.724+0000 7f222f200700 1 Processor -- start 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.724+0000 7f222f200700 1 -- start start 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.725+0000 7f222f200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:21.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.725+0000 7f222f200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f223007d840 con 0x7f223007ee40 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.725+0000 7f222de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.725+0000 7f222de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38664/0 (socket says 172.21.15.50:38664) 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.725+0000 7f222de00700 1 -- 172.21.15.50:0/3113295487 learned_addr learned my addr 172.21.15.50:0/3113295487 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.726+0000 7f222de00700 1 -- 172.21.15.50:0/3113295487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223007d9c0 con 0x7f223007ee40 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.726+0000 7f222de00700 1 --2- 172.21.15.50:0/3113295487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22200098a0 tx=0x7f2220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd73ae6a75b4c066 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.726+0000 7f222ca00700 1 -- 172.21.15.50:0/3113295487 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2220004160 con 0x7f223007ee40 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.726+0000 7f222ca00700 1 -- 172.21.15.50:0/3113295487 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2220004300 con 0x7f223007ee40 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.726+0000 7f222ca00700 1 -- 172.21.15.50:0/3113295487 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2220011af0 con 0x7f223007ee40 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.727+0000 7f222f200700 1 -- 172.21.15.50:0/3113295487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 msgr2=0x7f223007d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.727+0000 7f222f200700 1 --2- 172.21.15.50:0/3113295487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22200098a0 tx=0x7f2220009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.728+0000 7f222f200700 1 -- 172.21.15.50:0/3113295487 shutdown_connections 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.728+0000 7f222f200700 1 --2- 172.21.15.50:0/3113295487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f223007d270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.728+0000 7f222f200700 1 -- 172.21.15.50:0/3113295487 >> 172.21.15.50:0/3113295487 conn(0x7f2230105080 msgr2=0x7f2230107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:21.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.728+0000 7f222f200700 1 -- 172.21.15.50:0/3113295487 shutdown_connections 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.728+0000 7f222f200700 1 -- 172.21.15.50:0/3113295487 wait complete. 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.729+0000 7f222f200700 1 Processor -- start 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222f200700 1 -- start start 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222f200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222f200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2230110020 con 0x7f223007ee40 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38676/0 (socket says 172.21.15.50:38676) 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.730+0000 7f222de00700 1 -- 172.21.15.50:0/4218738876 learned_addr learned my addr 172.21.15.50:0/4218738876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f222de00700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22301a05e0 con 0x7f223007ee40 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f222de00700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22200117b0 tx=0x7f2220011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22200039c0 con 0x7f223007ee40 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2220003b60 con 0x7f223007ee40 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22200196a0 con 0x7f223007ee40 2024-05-10T20:55:21.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.731+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22301a0870 con 0x7f223007ee40 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.732+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22301a3480 con 0x7f223007ee40 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.733+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f222001a430 con 0x7f223007ee40 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.734+0000 7f2226a00700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.734+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2214041140 con 0x7f221403e520 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.734+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2220053c20 con 0x7f223007ee40 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.734+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.734+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.935+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:12.935+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:13.336+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:13.336+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:14.137+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:14.137+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:15.739+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:15.739+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:18.942+0000 7f222d400700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/484554811 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:18.942+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:19.367+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f2220023d10 con 0x7f223007ee40 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:19.367+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 msgr2=0x7f2214040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:19.367+0000 7f2226a00700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.371+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f2220055590 con 0x7f223007ee40 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.372+0000 7f2226a00700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 0x7f2214044280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:21.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.372+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22200521b0 con 0x7f2214041e60 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.374+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 0x7f2214044280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.377+0000 7f222d400700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 0x7f2214044280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f221c003a10 tx=0x7f221c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.379+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f22200521b0 con 0x7f2214041e60 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.390+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2230064e20 con 0x7f2214041e60 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.390+0000 7f2226a00700 1 -- 172.21.15.50:0/4218738876 <== mgr.14120 v2:172.21.15.50:6800/3321516664 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2230064e20 con 0x7f2214041e60 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 msgr2=0x7f2214044280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 0x7f2214044280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f221c003a10 tx=0x7f221c0092b0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 msgr2=0x7f22301a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:21.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22200117b0 tx=0x7f2220011b70 comp rx=0 tx=0).stop 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 shutdown_connections 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223007ee40 0x7f22301a00a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2214041e60 0x7f2214044280 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 --2- 172.21.15.50:0/4218738876 >> [v2:172.21.15.50:6800/484554811,v1:172.21.15.50:6801/484554811] conn(0x7f221403e520 0x7f2214040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 >> 172.21.15.50:0/4218738876 conn(0x7f2230105080 msgr2=0x7f2230105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.391+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 shutdown_connections 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:20.392+0000 7f222f200700 1 -- 172.21.15.50:0/4218738876 wait complete. 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 5 is available 2024-05-10T20:55:21.092 INFO:teuthology.orchestra.run.smithi050.stdout:Setting orchestrator backend to cephadm... 2024-05-10T20:55:21.648 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:21 smithi050 ceph-mon[29546]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T20:55:21.648 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:21 smithi050 ceph-mon[29546]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: [10/May/2024:20:55:21] ENGINE Bus STARTING 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: mgrmap e8: x(active, since 2s) 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: [10/May/2024:20:55:21] ENGINE Serving on http://172.21.15.50:8765 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: [10/May/2024:20:55:21] ENGINE Serving on https://172.21.15.50:7150 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: [10/May/2024:20:55:21] ENGINE Bus STARTED 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:22.761 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:22 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:22.780 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.803+0000 7fd528400700 1 Processor -- start 2024-05-10T20:55:22.780 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.803+0000 7fd528400700 1 -- start start 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.803+0000 7fd528400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.803+0000 7fd528400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd52010c800 con 0x7fd52010be00 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.804+0000 7fd525a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.804+0000 7fd525a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35872/0 (socket says 172.21.15.50:35872) 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.804+0000 7fd525a00700 1 -- 172.21.15.50:0/1057395945 learned_addr learned my addr 172.21.15.50:0/1057395945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.805+0000 7fd525a00700 1 -- 172.21.15.50:0/1057395945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52010cfd0 con 0x7fd52010be00 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.805+0000 7fd525a00700 1 --2- 172.21.15.50:0/1057395945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098a0 tx=0x7fd514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64f093da496f310 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.805+0000 7fd51fe00700 1 -- 172.21.15.50:0/1057395945 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514004160 con 0x7fd52010be00 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.805+0000 7fd51fe00700 1 -- 172.21.15.50:0/1057395945 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd514004300 con 0x7fd52010be00 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.805+0000 7fd51fe00700 1 -- 172.21.15.50:0/1057395945 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514011af0 con 0x7fd52010be00 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 -- 172.21.15.50:0/1057395945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 msgr2=0x7fd52010c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 --2- 172.21.15.50:0/1057395945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098a0 tx=0x7fd514009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:22.781 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 -- 172.21.15.50:0/1057395945 shutdown_connections 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 --2- 172.21.15.50:0/1057395945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd52010c210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 -- 172.21.15.50:0/1057395945 >> 172.21.15.50:0/1057395945 conn(0x7fd52007f3b0 msgr2=0x7fd520109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.806+0000 7fd528400700 1 -- 172.21.15.50:0/1057395945 shutdown_connections 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.807+0000 7fd528400700 1 -- 172.21.15.50:0/1057395945 wait complete. 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.808+0000 7fd528400700 1 Processor -- start 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.808+0000 7fd528400700 1 -- start start 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.808+0000 7fd528400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.808+0000 7fd528400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd520110380 con 0x7fd52010be00 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.809+0000 7fd525a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.809+0000 7fd525a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35880/0 (socket says 172.21.15.50:35880) 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.809+0000 7fd525a00700 1 -- 172.21.15.50:0/2573192203 learned_addr learned my addr 172.21.15.50:0/2573192203 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.809+0000 7fd525a00700 1 -- 172.21.15.50:0/2573192203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5201a0800 con 0x7fd52010be00 2024-05-10T20:55:22.782 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.809+0000 7fd525a00700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5140090e0 tx=0x7fd514011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.810+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5140039e0 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.810+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd514003b80 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.810+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514024690 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.810+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5201a0a90 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.810+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5201a36a0 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.811+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd520064e20 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.811+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd514022020 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.812+0000 7fd51e000700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 0x7fd5080409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.812+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5140536d0 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.812+0000 7fd525000700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 0x7fd5080409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.813+0000 7fd525000700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 0x7fd5080409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd510006fd0 tx=0x7fd510006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:21.818+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd514019ca0 con 0x7fd52010be00 2024-05-10T20:55:22.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.073+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd5201098e0 con 0x7fd50803e4c0 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.085+0000 7fd51e000700 1 -- 172.21.15.50:0/2573192203 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5201098e0 con 0x7fd50803e4c0 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 msgr2=0x7fd5080409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 0x7fd5080409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd510006fd0 tx=0x7fd510006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 msgr2=0x7fd5201a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5140090e0 tx=0x7fd514011940 comp rx=0 tx=0).stop 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 shutdown_connections 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd50803e4c0 0x7fd5080409d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 --2- 172.21.15.50:0/2573192203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52010be00 0x7fd5201a02c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 >> 172.21.15.50:0/2573192203 conn(0x7fd52007f3b0 msgr2=0x7fd520109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 shutdown_connections 2024-05-10T20:55:22.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:22.089+0000 7fd528400700 1 -- 172.21.15.50:0/2573192203 wait complete. 2024-05-10T20:55:23.725 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:23 smithi050 ceph-mon[29546]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:24.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.465+0000 7f1f5e600700 1 Processor -- start 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.465+0000 7f1f5e600700 1 -- start start 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.466+0000 7f1f5e600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.466+0000 7f1f5e600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c610 con 0x7f1f5810bc10 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.466+0000 7f1f5d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.466+0000 7f1f5d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35896/0 (socket says 172.21.15.50:35896) 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.466+0000 7f1f5d200700 1 -- 172.21.15.50:0/331181082 learned_addr learned my addr 172.21.15.50:0/331181082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:24.378 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.467+0000 7f1f5d200700 1 -- 172.21.15.50:0/331181082 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5810cde0 con 0x7f1f5810bc10 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.467+0000 7f1f5d200700 1 --2- 172.21.15.50:0/331181082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0098a0 tx=0x7f1f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6c544df277a8ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.467+0000 7f1f57400700 1 -- 172.21.15.50:0/331181082 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f4c004160 con 0x7f1f5810bc10 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.468+0000 7f1f57400700 1 -- 172.21.15.50:0/331181082 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f4c004300 con 0x7f1f5810bc10 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.468+0000 7f1f57400700 1 -- 172.21.15.50:0/331181082 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f4c011af0 con 0x7f1f5810bc10 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.468+0000 7f1f5e600700 1 -- 172.21.15.50:0/331181082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 msgr2=0x7f1f5810c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.468+0000 7f1f5e600700 1 --2- 172.21.15.50:0/331181082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0098a0 tx=0x7f1f4c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.469+0000 7f1f5e600700 1 -- 172.21.15.50:0/331181082 shutdown_connections 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.469+0000 7f1f5e600700 1 --2- 172.21.15.50:0/331181082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f5810c020 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:24.379 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.469+0000 7f1f5e600700 1 -- 172.21.15.50:0/331181082 >> 172.21.15.50:0/331181082 conn(0x7f1f5807f0e0 msgr2=0x7f1f58109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:24.383 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.469+0000 7f1f5e600700 1 -- 172.21.15.50:0/331181082 shutdown_connections 2024-05-10T20:55:24.383 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.469+0000 7f1f5e600700 1 -- 172.21.15.50:0/331181082 wait complete. 2024-05-10T20:55:24.383 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.470+0000 7f1f5e600700 1 Processor -- start 2024-05-10T20:55:24.383 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.470+0000 7f1f5e600700 1 -- start start 2024-05-10T20:55:24.383 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5e600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5e600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58110190 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35898/0 (socket says 172.21.15.50:35898) 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5d200700 1 -- 172.21.15.50:0/2940037880 learned_addr learned my addr 172.21.15.50:0/2940037880 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.471+0000 7f1f5d200700 1 -- 172.21.15.50:0/2940037880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f581a05d0 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f5d200700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011890 tx=0x7f1f4c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f4c0039e0 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f4c003b80 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581a0860 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f4c0246e0 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.472+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a3470 con 0x7f1f5810bc10 2024-05-10T20:55:24.384 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.473+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f58064e20 con 0x7f1f5810bc10 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.473+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1f4c022020 con 0x7f1f5810bc10 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.474+0000 7f1f55600700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 0x7f1f400409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.474+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f4c0537e0 con 0x7f1f5810bc10 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.474+0000 7f1f57e00700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 0x7f1f400409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.475+0000 7f1f57e00700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 0x7f1f400409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f44006fd0 tx=0x7f1f44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.480+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f4c053130 con 0x7f1f5810bc10 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.720+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1f581098e0 con 0x7f1f4003e4c0 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.722+0000 7f1f55600700 1 -- 172.21.15.50:0/2940037880 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1f581098e0 con 0x7f1f4003e4c0 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 msgr2=0x7f1f400409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 0x7f1f400409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f44006fd0 tx=0x7f1f44006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 msgr2=0x7f1f581a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c011890 tx=0x7f1f4c011970 comp rx=0 tx=0).stop 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 shutdown_connections 2024-05-10T20:55:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f1f4003e4c0 0x7f1f400409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 --2- 172.21.15.50:0/2940037880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f5810bc10 0x7f1f581a0090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.726+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 >> 172.21.15.50:0/2940037880 conn(0x7f1f5807f0e0 msgr2=0x7f1f581091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.727+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 shutdown_connections 2024-05-10T20:55:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:23.727+0000 7f1f5e600700 1 -- 172.21.15.50:0/2940037880 wait complete. 2024-05-10T20:55:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:Generating ssh key... 2024-05-10T20:55:24.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:24 smithi050 ceph-mon[29546]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:25.674 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: Generating public/private rsa key pair. 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: Your identification has been saved in /tmp/tmpk7ek62sm/key. 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: Your public key has been saved in /tmp/tmpk7ek62sm/key.pub. 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: The key fingerprint is: 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: SHA256:EbxQ8DZRQInoEZNiG6Q8ITRpNAH6iOICCCltPsk+Cqw ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: The key's randomart image is: 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: +---[RSA 3072]----+ 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |*O+ o+.**+. | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |+*B +.o.oo | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |B+o= . .=. | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |=*o.. ..o | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |* * S | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |=. . | 2024-05-10T20:55:25.675 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |ooo | 2024-05-10T20:55:25.676 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |+. . | 2024-05-10T20:55:25.676 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: |E | 2024-05-10T20:55:25.676 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: +----[SHA256]-----+ 2024-05-10T20:55:26.049 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.048+0000 7f2372600700 1 Processor -- start 2024-05-10T20:55:26.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.048+0000 7f2372600700 1 -- start start 2024-05-10T20:55:26.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.048+0000 7f2372600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:26.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.048+0000 7f2372600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f236c07e0d0 con 0x7f236c07f6d0 2024-05-10T20:55:26.050 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.049+0000 7f236b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:26.055 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.049+0000 7f236b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58576/0 (socket says 172.21.15.50:58576) 2024-05-10T20:55:26.055 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.049+0000 7f236b400700 1 -- 172.21.15.50:0/11624864 learned_addr learned my addr 172.21.15.50:0/11624864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:26.055 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.049+0000 7f236b400700 1 -- 172.21.15.50:0/11624864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c07e250 con 0x7f236c07f6d0 2024-05-10T20:55:26.055 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.050+0000 7f236b400700 1 --2- 172.21.15.50:0/11624864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae31a35940f28298 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.050+0000 7f236aa00700 1 -- 172.21.15.50:0/11624864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c004160 con 0x7f236c07f6d0 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.050+0000 7f236aa00700 1 -- 172.21.15.50:0/11624864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f235c004300 con 0x7f236c07f6d0 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.050+0000 7f236aa00700 1 -- 172.21.15.50:0/11624864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c011af0 con 0x7f236c07f6d0 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 -- 172.21.15.50:0/11624864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 msgr2=0x7f236c07db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 --2- 172.21.15.50:0/11624864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 -- 172.21.15.50:0/11624864 shutdown_connections 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 --2- 172.21.15.50:0/11624864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c07db00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 -- 172.21.15.50:0/11624864 >> 172.21.15.50:0/11624864 conn(0x7f236c105380 msgr2=0x7f236c1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 -- 172.21.15.50:0/11624864 shutdown_connections 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.051+0000 7f2372600700 1 -- 172.21.15.50:0/11624864 wait complete. 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.052+0000 7f2372600700 1 Processor -- start 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.052+0000 7f2372600700 1 -- start start 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f2372600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:26.056 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f2372600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f236c110200 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f236b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f236b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58590/0 (socket says 172.21.15.50:58590) 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f236b400700 1 -- 172.21.15.50:0/1185712101 learned_addr learned my addr 172.21.15.50:0/1185712101 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.053+0000 7f236b400700 1 -- 172.21.15.50:0/1185712101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c1a07c0 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f236b400700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f235c0117b0 tx=0x7f235c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c0039c0 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f235c003b60 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c1a0a50 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c0196a0 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.054+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a3660 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.055+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c064e20 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.056+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f235c01a430 con 0x7f236c07f6d0 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.056+0000 7f2368c00700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 0x7f2350040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:26.057 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.056+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f235c053bc0 con 0x7f236c07f6d0 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.056+0000 7f2363e00700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 0x7f2350040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.057+0000 7f2363e00700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 0x7f2350040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2354006fd0 tx=0x7f2354006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.062+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f235c00b3e0 con 0x7f236c07f6d0 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.301+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f236c107510 con 0x7f235003e470 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.401+0000 7f2368c00700 1 -- 172.21.15.50:0/1185712101 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f236c107510 con 0x7f235003e470 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 msgr2=0x7f2350040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 0x7f2350040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2354006fd0 tx=0x7f2354006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 msgr2=0x7f236c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f235c0117b0 tx=0x7f235c011b70 comp rx=0 tx=0).stop 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 shutdown_connections 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f235003e470 0x7f2350040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 --2- 172.21.15.50:0/1185712101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f236c07f6d0 0x7f236c1a0280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.405+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 >> 172.21.15.50:0/1185712101 conn(0x7f236c105380 msgr2=0x7f236c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:26.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.406+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 shutdown_connections 2024-05-10T20:55:26.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:25.406+0000 7f2372600700 1 -- 172.21.15.50:0/1185712101 wait complete. 2024-05-10T20:55:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:26 smithi050 ceph-mon[29546]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:26 smithi050 ceph-mon[29546]: Generating ssh key... 2024-05-10T20:55:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:26 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:26 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:27.677 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:55:27.677 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.748+0000 7fd841600700 1 Processor -- start 2024-05-10T20:55:27.677 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.749+0000 7fd841600700 1 -- start start 2024-05-10T20:55:27.677 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.749+0000 7fd841600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:27.678 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.749+0000 7fd841600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10c610 con 0x7fd83c10bc10 2024-05-10T20:55:27.678 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.750+0000 7fd83be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:27.678 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.750+0000 7fd83be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58604/0 (socket says 172.21.15.50:58604) 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.750+0000 7fd83be00700 1 -- 172.21.15.50:0/1133651167 learned_addr learned my addr 172.21.15.50:0/1133651167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.750+0000 7fd83be00700 1 -- 172.21.15.50:0/1133651167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c10cde0 con 0x7fd83c10bc10 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.751+0000 7fd83be00700 1 --2- 172.21.15.50:0/1133651167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0098a0 tx=0x7fd82c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1244ded6875a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.751+0000 7fd83aa00700 1 -- 172.21.15.50:0/1133651167 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82c004160 con 0x7fd83c10bc10 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.751+0000 7fd83aa00700 1 -- 172.21.15.50:0/1133651167 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd82c004300 con 0x7fd83c10bc10 2024-05-10T20:55:27.679 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.751+0000 7fd83aa00700 1 -- 172.21.15.50:0/1133651167 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82c011af0 con 0x7fd83c10bc10 2024-05-10T20:55:27.680 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.752+0000 7fd841600700 1 -- 172.21.15.50:0/1133651167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 msgr2=0x7fd83c10c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:27.680 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.752+0000 7fd841600700 1 --2- 172.21.15.50:0/1133651167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0098a0 tx=0x7fd82c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:27.680 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.752+0000 7fd841600700 1 -- 172.21.15.50:0/1133651167 shutdown_connections 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.752+0000 7fd841600700 1 --2- 172.21.15.50:0/1133651167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c10c020 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.752+0000 7fd841600700 1 -- 172.21.15.50:0/1133651167 >> 172.21.15.50:0/1133651167 conn(0x7fd83c07f0e0 msgr2=0x7fd83c109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.753+0000 7fd841600700 1 -- 172.21.15.50:0/1133651167 shutdown_connections 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.753+0000 7fd841600700 1 -- 172.21.15.50:0/1133651167 wait complete. 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.754+0000 7fd841600700 1 Processor -- start 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.754+0000 7fd841600700 1 -- start start 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.754+0000 7fd841600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:27.681 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.754+0000 7fd841600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c110190 con 0x7fd83c10bc10 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.755+0000 7fd83be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.755+0000 7fd83be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58614/0 (socket says 172.21.15.50:58614) 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.755+0000 7fd83be00700 1 -- 172.21.15.50:0/1116227244 learned_addr learned my addr 172.21.15.50:0/1116227244 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.755+0000 7fd83be00700 1 -- 172.21.15.50:0/1116227244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c1a05f0 con 0x7fd83c10bc10 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.755+0000 7fd83be00700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd82c011890 tx=0x7fd82c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.756+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82c0039e0 con 0x7fd83c10bc10 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.756+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd82c003b80 con 0x7fd83c10bc10 2024-05-10T20:55:27.683 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.756+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82c0246e0 con 0x7fd83c10bc10 2024-05-10T20:55:27.684 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.756+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c1a0880 con 0x7fd83c10bc10 2024-05-10T20:55:27.684 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.756+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c1a3490 con 0x7fd83c10bc10 2024-05-10T20:55:27.684 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.757+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c064e20 con 0x7fd83c10bc10 2024-05-10T20:55:27.684 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.757+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd82c022020 con 0x7fd83c10bc10 2024-05-10T20:55:27.684 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.758+0000 7fd838c00700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 0x7fd8200409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:27.685 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.758+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd82c053700 con 0x7fd83c10bc10 2024-05-10T20:55:27.685 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.758+0000 7fd83b400700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 0x7fd8200409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.759+0000 7fd83b400700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 0x7fd8200409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd828006fd0 tx=0x7fd828006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:26.764+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd82c0212c0 con 0x7fd83c10bc10 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.002+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd83c109920 con 0x7fd82003e4c0 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.005+0000 7fd838c00700 1 -- 172.21.15.50:0/1116227244 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd83c109920 con 0x7fd82003e4c0 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 msgr2=0x7fd8200409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 0x7fd8200409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd828006fd0 tx=0x7fd828006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 msgr2=0x7fd83c1a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd82c011890 tx=0x7fd82c011970 comp rx=0 tx=0).stop 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 shutdown_connections 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fd82003e4c0 0x7fd8200409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 --2- 172.21.15.50:0/1116227244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd83c10bc10 0x7fd83c1a00b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:27.687 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 >> 172.21.15.50:0/1116227244 conn(0x7fd83c07f0e0 msgr2=0x7fd83c1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:27.688 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 shutdown_connections 2024-05-10T20:55:27.688 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:27.008+0000 7fd841600700 1 -- 172.21.15.50:0/1116227244 wait complete. 2024-05-10T20:55:27.688 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T20:55:27.688 INFO:teuthology.orchestra.run.smithi050.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T20:55:27.690 INFO:teuthology.orchestra.run.smithi050.stdout:Adding host smithi050... 2024-05-10T20:55:27.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:27 smithi050 ceph-mon[29546]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:28.910 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:28 smithi050 ceph-mon[29546]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:31.100 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:30 smithi050 ceph-mon[29546]: Deploying cephadm binary to smithi050 2024-05-10T20:55:35.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Added host 'smithi050' with addr '172.21.15.50' 2024-05-10T20:55:35.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.392+0000 7f2daa000700 1 Processor -- start 2024-05-10T20:55:35.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.392+0000 7f2daa000700 1 -- start start 2024-05-10T20:55:35.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.393+0000 7f2daa000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:35.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.393+0000 7f2daa000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2da407dfe0 con 0x7f2da407f670 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.393+0000 7f2da3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.394+0000 7f2da3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58618/0 (socket says 172.21.15.50:58618) 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.394+0000 7f2da3400700 1 -- 172.21.15.50:0/1632937807 learned_addr learned my addr 172.21.15.50:0/1632937807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.394+0000 7f2da3400700 1 -- 172.21.15.50:0/1632937807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da407e160 con 0x7f2da407f670 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.394+0000 7f2da3400700 1 --2- 172.21.15.50:0/1632937807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6c1a1e06a0ae65 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.395+0000 7f2da2000700 1 -- 172.21.15.50:0/1632937807 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94004160 con 0x7f2da407f670 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.395+0000 7f2da2000700 1 -- 172.21.15.50:0/1632937807 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d94004300 con 0x7f2da407f670 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.395+0000 7f2da2000700 1 -- 172.21.15.50:0/1632937807 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94011af0 con 0x7f2da407f670 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.395+0000 7f2daa000700 1 -- 172.21.15.50:0/1632937807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 msgr2=0x7f2da407daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.395+0000 7f2daa000700 1 --2- 172.21.15.50:0/1632937807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.396+0000 7f2daa000700 1 -- 172.21.15.50:0/1632937807 shutdown_connections 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.396+0000 7f2daa000700 1 --2- 172.21.15.50:0/1632937807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da407daa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:35.887 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.396+0000 7f2daa000700 1 -- 172.21.15.50:0/1632937807 >> 172.21.15.50:0/1632937807 conn(0x7f2da4105380 msgr2=0x7f2da41077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.396+0000 7f2daa000700 1 -- 172.21.15.50:0/1632937807 shutdown_connections 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.396+0000 7f2daa000700 1 -- 172.21.15.50:0/1632937807 wait complete. 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.397+0000 7f2daa000700 1 Processor -- start 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.397+0000 7f2daa000700 1 -- start start 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.397+0000 7f2daa000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.397+0000 7f2daa000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4110200 con 0x7f2da407f670 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.398+0000 7f2da3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.398+0000 7f2da3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58630/0 (socket says 172.21.15.50:58630) 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.398+0000 7f2da3400700 1 -- 172.21.15.50:0/486690152 learned_addr learned my addr 172.21.15.50:0/486690152 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.398+0000 7f2da3400700 1 -- 172.21.15.50:0/486690152 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41a0790 con 0x7f2da407f670 2024-05-10T20:55:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2da3400700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fb0 tx=0x7f2d94011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94011bc0 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d94024440 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d9401a5c0 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41a0a20 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.399+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a3630 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.400+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2d9401a830 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.400+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4064e20 con 0x7f2da407f670 2024-05-10T20:55:35.889 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.401+0000 7f2d9be00700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 0x7f2d88040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:35.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.401+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d940537c0 con 0x7f2da407f670 2024-05-10T20:55:35.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.401+0000 7f2da2a00700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 0x7f2d88040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:35.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.402+0000 7f2da2a00700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 0x7f2d88040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:35.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.407+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d9400b3e0 con 0x7f2da407f670 2024-05-10T20:55:35.890 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:28.650+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}) -- 0x7f2da4107520 con 0x7f2d8803e470 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.111+0000 7f2d9be00700 1 -- 172.21.15.50:0/486690152 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2da4107520 con 0x7f2d8803e470 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.115+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 msgr2=0x7f2d88040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 0x7f2d88040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 msgr2=0x7f2da41a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fb0 tx=0x7f2d94011890 comp rx=0 tx=0).stop 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 shutdown_connections 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f2d8803e470 0x7f2d88040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 --2- 172.21.15.50:0/486690152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da407f670 0x7f2da41a0250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 >> 172.21.15.50:0/486690152 conn(0x7f2da4105380 msgr2=0x7f2da4106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 shutdown_connections 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:35.116+0000 7f2daa000700 1 -- 172.21.15.50:0/486690152 wait complete. 2024-05-10T20:55:35.891 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mon service... 2024-05-10T20:55:36.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:36 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:36.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:36 smithi050 ceph-mon[29546]: Added host smithi050 2024-05-10T20:55:36.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:36 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:37.739 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T20:55:37.739 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.555+0000 7f9b37800700 1 Processor -- start 2024-05-10T20:55:37.739 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.555+0000 7f9b37800700 1 -- start start 2024-05-10T20:55:37.739 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.556+0000 7f9b37800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:37.740 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.556+0000 7f9b37800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3007ce10 con 0x7f9b3010a750 2024-05-10T20:55:37.740 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.556+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:37.740 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.557+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35700/0 (socket says 172.21.15.50:35700) 2024-05-10T20:55:37.740 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.557+0000 7f9b34e00700 1 -- 172.21.15.50:0/3813383201 learned_addr learned my addr 172.21.15.50:0/3813383201 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:37.741 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.557+0000 7f9b34e00700 1 -- 172.21.15.50:0/3813383201 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3010d0b0 con 0x7f9b3010a750 2024-05-10T20:55:37.741 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.558+0000 7f9b34e00700 1 --2- 172.21.15.50:0/3813383201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098a0 tx=0x7f9b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15607e7e8ea0270 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:37.741 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.558+0000 7f9b2f400700 1 -- 172.21.15.50:0/3813383201 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24004160 con 0x7f9b3010a750 2024-05-10T20:55:37.741 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.558+0000 7f9b2f400700 1 -- 172.21.15.50:0/3813383201 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b24004300 con 0x7f9b3010a750 2024-05-10T20:55:37.741 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.558+0000 7f9b2f400700 1 -- 172.21.15.50:0/3813383201 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24011af0 con 0x7f9b3010a750 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 -- 172.21.15.50:0/3813383201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 msgr2=0x7f9b3010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 --2- 172.21.15.50:0/3813383201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098a0 tx=0x7f9b24009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 -- 172.21.15.50:0/3813383201 shutdown_connections 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 --2- 172.21.15.50:0/3813383201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b3010cb70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 -- 172.21.15.50:0/3813383201 >> 172.21.15.50:0/3813383201 conn(0x7f9b301041d0 msgr2=0x7f9b30106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 -- 172.21.15.50:0/3813383201 shutdown_connections 2024-05-10T20:55:37.743 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.559+0000 7f9b37800700 1 -- 172.21.15.50:0/3813383201 wait complete. 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.560+0000 7f9b37800700 1 Processor -- start 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.560+0000 7f9b37800700 1 -- start start 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b37800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b37800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010dfa0 con 0x7f9b3010a750 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:37.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35706/0 (socket says 172.21.15.50:35706) 2024-05-10T20:55:37.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b34e00700 1 -- 172.21.15.50:0/2051909681 learned_addr learned my addr 172.21.15.50:0/2051909681 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:37.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.561+0000 7f9b34e00700 1 -- 172.21.15.50:0/2051909681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3010a410 con 0x7f9b3010a750 2024-05-10T20:55:37.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b34e00700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009fb0 tx=0x7f9b24011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:37.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24011bc0 con 0x7f9b3010a750 2024-05-10T20:55:37.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b24024440 con 0x7f9b3010a750 2024-05-10T20:55:37.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b2401a5c0 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b30108660 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.562+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b30108b40 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.564+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b30064e20 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.564+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9b240038b0 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.564+0000 7f9b2d600700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 0x7f9b18040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.564+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b24053830 con 0x7f9b3010a750 2024-05-10T20:55:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.564+0000 7f9b2fe00700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 0x7f9b18040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:37.749 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.565+0000 7f9b2fe00700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 0x7f9b18040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.570+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b2400f2e0 con 0x7f9b3010a750 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.822+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9b30109330 con 0x7f9b1803e470 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.832+0000 7f9b2d600700 1 -- 172.21.15.50:0/2051909681 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9b30109330 con 0x7f9b1803e470 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.835+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 msgr2=0x7f9b18040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 0x7f9b18040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 msgr2=0x7f9b30109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009fb0 tx=0x7f9b24011890 comp rx=0 tx=0).stop 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 shutdown_connections 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f9b1803e470 0x7f9b18040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 --2- 172.21.15.50:0/2051909681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b3010a750 0x7f9b30109ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:37.750 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 >> 172.21.15.50:0/2051909681 conn(0x7f9b301041d0 msgr2=0x7f9b30105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:37.751 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.836+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 shutdown_connections 2024-05-10T20:55:37.751 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:36.837+0000 7f9b37800700 1 -- 172.21.15.50:0/2051909681 wait complete. 2024-05-10T20:55:37.751 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mgr service... 2024-05-10T20:55:37.998 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:37 smithi050 ceph-mon[29546]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:37.999 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:37 smithi050 ceph-mon[29546]: Saving service mon spec with placement count:5 2024-05-10T20:55:37.999 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:37 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:39.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T20:55:39.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.769+0000 7f0c7fa00700 1 Processor -- start 2024-05-10T20:55:39.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.770+0000 7f0c7fa00700 1 -- start start 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.770+0000 7f0c7fa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.770+0000 7f0c7fa00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7807d770 con 0x7f0c7807ee00 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.770+0000 7f0c7d000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.771+0000 7f0c7d000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35710/0 (socket says 172.21.15.50:35710) 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.771+0000 7f0c7d000700 1 -- 172.21.15.50:0/1638890061 learned_addr learned my addr 172.21.15.50:0/1638890061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.771+0000 7f0c7d000700 1 -- 172.21.15.50:0/1638890061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7807d8f0 con 0x7f0c7807ee00 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.771+0000 7f0c7d000700 1 --2- 172.21.15.50:0/1638890061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0098a0 tx=0x7f0c6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647c98a0ce227fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.772+0000 7f0c77400700 1 -- 172.21.15.50:0/1638890061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6c004160 con 0x7f0c7807ee00 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.772+0000 7f0c77400700 1 -- 172.21.15.50:0/1638890061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0c6c004300 con 0x7f0c7807ee00 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.772+0000 7f0c77400700 1 -- 172.21.15.50:0/1638890061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6c011af0 con 0x7f0c7807ee00 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 -- 172.21.15.50:0/1638890061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 msgr2=0x7f0c7807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/1638890061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0098a0 tx=0x7f0c6c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:39.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 -- 172.21.15.50:0/1638890061 shutdown_connections 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/1638890061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c7807d230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 -- 172.21.15.50:0/1638890061 >> 172.21.15.50:0/1638890061 conn(0x7f0c78105210 msgr2=0x7f0c781076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 -- 172.21.15.50:0/1638890061 shutdown_connections 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.773+0000 7f0c7fa00700 1 -- 172.21.15.50:0/1638890061 wait complete. 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.774+0000 7f0c7fa00700 1 Processor -- start 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.774+0000 7f0c7fa00700 1 -- start start 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7fa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7fa00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781101b0 con 0x7f0c7807ee00 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7d000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7d000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35712/0 (socket says 172.21.15.50:35712) 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7d000700 1 -- 172.21.15.50:0/2684489484 learned_addr learned my addr 172.21.15.50:0/2684489484 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.775+0000 7f0c7d000700 1 -- 172.21.15.50:0/2684489484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a0780 con 0x7f0c7807ee00 2024-05-10T20:55:39.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c7d000700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0117b0 tx=0x7f0c6c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6c0039c0 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a0a10 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0c6c003b60 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6c0196a0 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.776+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a3620 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.778+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78064e20 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.778+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0c6c01ab00 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.779+0000 7f0c75600700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 0x7f0c60040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.779+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0c6c053ba0 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.779+0000 7f0c77e00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 0x7f0c60040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.780+0000 7f0c77e00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 0x7f0c60040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006fd0 tx=0x7f0c64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:38.784+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c6c0534f0 con 0x7f0c7807ee00 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.045+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0c781063c0 con 0x7f0c6003e470 2024-05-10T20:55:39.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.052+0000 7f0c75600700 1 -- 172.21.15.50:0/2684489484 <== mgr.14120 v2:172.21.15.50:6800/3321516664 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0c781063c0 con 0x7f0c6003e470 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 msgr2=0x7f0c60040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 0x7f0c60040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c64006fd0 tx=0x7f0c64006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 msgr2=0x7f0c781a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0117b0 tx=0x7f0c6c011b70 comp rx=0 tx=0).stop 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 shutdown_connections 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f0c6003e470 0x7f0c60040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 --2- 172.21.15.50:0/2684489484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c7807ee00 0x7f0c781a0240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 >> 172.21.15.50:0/2684489484 conn(0x7f0c78105210 msgr2=0x7f0c78105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 shutdown_connections 2024-05-10T20:55:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:39.056+0000 7f0c7fa00700 1 -- 172.21.15.50:0/2684489484 wait complete. 2024-05-10T20:55:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:40 smithi050 ceph-mon[29546]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:40 smithi050 ceph-mon[29546]: Saving service mgr spec with placement count:2 2024-05-10T20:55:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:40 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:40 smithi050 ceph-mon[29546]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:55:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:40 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:41.582 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:41 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:41.582 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:41 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3883632608' entity='client.admin' 2024-05-10T20:55:41.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.601+0000 7f5bf9000700 1 Processor -- start 2024-05-10T20:55:41.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.601+0000 7f5bf9000700 1 -- start start 2024-05-10T20:55:41.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.601+0000 7f5bf9000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.601+0000 7f5bf9000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf407d770 con 0x7f5bf407ee00 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.602+0000 7f5bf1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.602+0000 7f5bf1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35720/0 (socket says 172.21.15.50:35720) 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.602+0000 7f5bf1e00700 1 -- 172.21.15.50:0/3064802927 learned_addr learned my addr 172.21.15.50:0/3064802927 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.603+0000 7f5bf1e00700 1 -- 172.21.15.50:0/3064802927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf407d8f0 con 0x7f5bf407ee00 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.603+0000 7f5bf1e00700 1 --2- 172.21.15.50:0/3064802927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098a0 tx=0x7f5be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bc5764092abd09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.603+0000 7f5bf1400700 1 -- 172.21.15.50:0/3064802927 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be4004160 con 0x7f5bf407ee00 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.603+0000 7f5bf1400700 1 -- 172.21.15.50:0/3064802927 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4004300 con 0x7f5bf407ee00 2024-05-10T20:55:41.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.603+0000 7f5bf1400700 1 -- 172.21.15.50:0/3064802927 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be4011a20 con 0x7f5bf407ee00 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.604+0000 7f5bf9000700 1 -- 172.21.15.50:0/3064802927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 msgr2=0x7f5bf407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.604+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3064802927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098a0 tx=0x7f5be4009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.605+0000 7f5bf9000700 1 -- 172.21.15.50:0/3064802927 shutdown_connections 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.605+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3064802927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf407d230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.605+0000 7f5bf9000700 1 -- 172.21.15.50:0/3064802927 >> 172.21.15.50:0/3064802927 conn(0x7f5bf4105210 msgr2=0x7f5bf41076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.605+0000 7f5bf9000700 1 -- 172.21.15.50:0/3064802927 shutdown_connections 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.605+0000 7f5bf9000700 1 -- 172.21.15.50:0/3064802927 wait complete. 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.606+0000 7f5bf9000700 1 Processor -- start 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.606+0000 7f5bf9000700 1 -- start start 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.606+0000 7f5bf9000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.606+0000 7f5bf9000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41101f0 con 0x7f5bf407ee00 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.607+0000 7f5bf1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.607+0000 7f5bf1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35724/0 (socket says 172.21.15.50:35724) 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.607+0000 7f5bf1e00700 1 -- 172.21.15.50:0/3883632608 learned_addr learned my addr 172.21.15.50:0/3883632608 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.607+0000 7f5bf1e00700 1 -- 172.21.15.50:0/3883632608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf41a0710 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5bf1e00700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009520 tx=0x7f5be4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be40039e0 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be401b8f0 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf41a09a0 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be40197d0 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.608+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf41a35b0 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.609+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf4064e20 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.610+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5be4019a40 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.610+0000 7f5beb400700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 0x7f5bd80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.610+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5be4053f20 con 0x7f5bf407ee00 2024-05-10T20:55:41.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.611+0000 7f5be9400700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 0x7f5bd80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.611+0000 7f5be9400700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 0x7f5bd80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc006fd0 tx=0x7f5bdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.616+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5be401f030 con 0x7f5bf407ee00 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.854+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f5bf41a3bb0 con 0x7f5bf407ee00 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.859+0000 7f5beb400700 1 -- 172.21.15.50:0/3883632608 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f5be401bda0 con 0x7f5bf407ee00 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 msgr2=0x7f5bd80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 0x7f5bd80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc006fd0 tx=0x7f5bdc006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 msgr2=0x7f5bf41a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009520 tx=0x7f5be4011aa0 comp rx=0 tx=0).stop 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 shutdown_connections 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f5bd803e4c0 0x7f5bd80409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 --2- 172.21.15.50:0/3883632608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf407ee00 0x7f5bf41a01d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.863+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 >> 172.21.15.50:0/3883632608 conn(0x7f5bf4105210 msgr2=0x7f5bf4105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.864+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 shutdown_connections 2024-05-10T20:55:41.599 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:40.864+0000 7f5bf9000700 1 -- 172.21.15.50:0/3883632608 wait complete. 2024-05-10T20:55:42.524 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:42 smithi050 ceph-mon[29546]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:55:43.480 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.263+0000 7f428f200700 1 Processor -- start 2024-05-10T20:55:43.480 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.263+0000 7f428f200700 1 -- start start 2024-05-10T20:55:43.480 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.264+0000 7f428f200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:43.480 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.264+0000 7f428f200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4288112210 con 0x7f4288111830 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.265+0000 7f4287e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.265+0000 7f4287e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35736/0 (socket says 172.21.15.50:35736) 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.265+0000 7f4287e00700 1 -- 172.21.15.50:0/3496808065 learned_addr learned my addr 172.21.15.50:0/3496808065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.265+0000 7f4287e00700 1 -- 172.21.15.50:0/3496808065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288112390 con 0x7f4288111830 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.266+0000 7f4287e00700 1 --2- 172.21.15.50:0/3496808065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42780098a0 tx=0x7f4278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2ab3fa7471f5e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.266+0000 7f4286a00700 1 -- 172.21.15.50:0/3496808065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4278004160 con 0x7f4288111830 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.266+0000 7f4286a00700 1 -- 172.21.15.50:0/3496808065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004300 con 0x7f4288111830 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.266+0000 7f4286a00700 1 -- 172.21.15.50:0/3496808065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4278011a20 con 0x7f4288111830 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.267+0000 7f428f200700 1 -- 172.21.15.50:0/3496808065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 msgr2=0x7f4288111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.267+0000 7f428f200700 1 --2- 172.21.15.50:0/3496808065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42780098a0 tx=0x7f4278009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.267+0000 7f428f200700 1 -- 172.21.15.50:0/3496808065 shutdown_connections 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.267+0000 7f428f200700 1 --2- 172.21.15.50:0/3496808065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f4288111c40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.267+0000 7f428f200700 1 -- 172.21.15.50:0/3496808065 >> 172.21.15.50:0/3496808065 conn(0x7f428807f3b0 msgr2=0x7f4288081800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.268+0000 7f428f200700 1 -- 172.21.15.50:0/3496808065 shutdown_connections 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.268+0000 7f428f200700 1 -- 172.21.15.50:0/3496808065 wait complete. 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.269+0000 7f428f200700 1 Processor -- start 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.269+0000 7f428f200700 1 -- start start 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.269+0000 7f428f200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.269+0000 7f428f200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4288114790 con 0x7f4288111830 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.270+0000 7f4287e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.270+0000 7f4287e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35742/0 (socket says 172.21.15.50:35742) 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.270+0000 7f4287e00700 1 -- 172.21.15.50:0/2833128884 learned_addr learned my addr 172.21.15.50:0/2833128884 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.270+0000 7f4287e00700 1 -- 172.21.15.50:0/2833128884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42881a4b80 con 0x7f4288111830 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f4287e00700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42780119f0 tx=0x7f4278011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4278003a60 con 0x7f4288111830 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427801b8f0 con 0x7f4288111830 2024-05-10T20:55:43.482 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42881a4e10 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4278023880 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.271+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42881a7a20 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.272+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4288064e20 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.273+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f427801b440 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.274+0000 7f4284c00700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 0x7f426c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.274+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4278053ee0 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.274+0000 7f4287400700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 0x7f426c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.275+0000 7f4287400700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 0x7f426c0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.280+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f427801f030 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.519+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f42881a7d30 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.524+0000 7f4284c00700 1 -- 172.21.15.50:0/2833128884 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4288112390 con 0x7f4288111830 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 msgr2=0x7f426c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 0x7f426c0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:43.483 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 msgr2=0x7f42881a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42780119f0 tx=0x7f4278011ad0 comp rx=0 tx=0).stop 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 shutdown_connections 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f426c03e4c0 0x7f426c0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 --2- 172.21.15.50:0/2833128884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4288111830 0x7f42881a4640 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.527+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 >> 172.21.15.50:0/2833128884 conn(0x7f428807f3b0 msgr2=0x7f4288080cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.528+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 shutdown_connections 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:42.528+0000 7f428f200700 1 -- 172.21.15.50:0/2833128884 wait complete. 2024-05-10T20:55:43.484 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling the dashboard module... 2024-05-10T20:55:43.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:43 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2833128884' entity='client.admin' 2024-05-10T20:55:43.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:43 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:44.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:44 smithi050 ceph-mon[29546]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:55:45.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:45 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2987658523' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T20:55:45.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:45 smithi050 ceph-mon[29546]: from='mgr.14120 172.21.15.50:0/2162255015' entity='mgr.x' 2024-05-10T20:55:46.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: ignoring --setuser ceph since I am not root 2024-05-10T20:55:46.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: ignoring --setgroup ceph since I am not root 2024-05-10T20:55:46.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:45.942+0000 7fc3cf3ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T20:55:46.251 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.197+0000 7f9663a00700 1 Processor -- start 2024-05-10T20:55:46.257 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.197+0000 7f9663a00700 1 -- start start 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.198+0000 7f9663a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.198+0000 7f9663a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f965c10a600 con 0x7f965c109c00 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.198+0000 7f9661000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.198+0000 7f9661000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42216/0 (socket says 172.21.15.50:42216) 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.198+0000 7f9661000700 1 -- 172.21.15.50:0/17737285 learned_addr learned my addr 172.21.15.50:0/17737285 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.199+0000 7f9661000700 1 -- 172.21.15.50:0/17737285 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c10ad80 con 0x7f965c109c00 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.199+0000 7f9661000700 1 --2- 172.21.15.50:0/17737285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe0de0c28a2b298 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.199+0000 7f965b400700 1 -- 172.21.15.50:0/17737285 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9650004160 con 0x7f965c109c00 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.199+0000 7f965b400700 1 -- 172.21.15.50:0/17737285 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9650004300 con 0x7f965c109c00 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.199+0000 7f965b400700 1 -- 172.21.15.50:0/17737285 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9650011a20 con 0x7f965c109c00 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.200+0000 7f9663a00700 1 -- 172.21.15.50:0/17737285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 msgr2=0x7f965c10a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.200+0000 7f9663a00700 1 --2- 172.21.15.50:0/17737285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.200+0000 7f9663a00700 1 -- 172.21.15.50:0/17737285 shutdown_connections 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.200+0000 7f9663a00700 1 --2- 172.21.15.50:0/17737285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c10a010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.200+0000 7f9663a00700 1 -- 172.21.15.50:0/17737285 >> 172.21.15.50:0/17737285 conn(0x7f965c105240 msgr2=0x7f965c1076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.201+0000 7f9663a00700 1 -- 172.21.15.50:0/17737285 shutdown_connections 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.201+0000 7f9663a00700 1 -- 172.21.15.50:0/17737285 wait complete. 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.202+0000 7f9663a00700 1 Processor -- start 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.202+0000 7f9663a00700 1 -- start start 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9663a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9663a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f965c10e130 con 0x7f965c109c00 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9661000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9661000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42228/0 (socket says 172.21.15.50:42228) 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9661000700 1 -- 172.21.15.50:0/2987658523 learned_addr learned my addr 172.21.15.50:0/2987658523 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:46.259 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.203+0000 7f9661000700 1 -- 172.21.15.50:0/2987658523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c07f9d0 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9661000700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9650009130 tx=0x7f9650011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9650003aa0 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965001a8e0 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c07dc20 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96500248f0 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.204+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c07e100 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.206+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c064e20 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.207+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9650022020 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.207+0000 7f9659600700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 0x7f96440409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.207+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f96500537a0 con 0x7f965c109c00 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.208+0000 7f965be00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 0x7f96440409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.209+0000 7f965be00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 0x7f96440409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9648006fd0 tx=0x7f9648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:46.260 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.214+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9650029df0 con 0x7f965c109c00 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:44.533+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f965c07e7f0 con 0x7f965c109c00 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.533+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f965001aa90 con 0x7f965c109c00 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.537+0000 7f9659600700 1 -- 172.21.15.50:0/2987658523 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f9650055ca0 con 0x7f965c109c00 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 msgr2=0x7f96440409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 0x7f96440409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9648006fd0 tx=0x7f9648006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 msgr2=0x7f965c07f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9650009130 tx=0x7f9650011aa0 comp rx=0 tx=0).stop 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 shutdown_connections 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7f964403e4c0 0x7f96440409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 --2- 172.21.15.50:0/2987658523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f965c109c00 0x7f965c07f490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.538+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 >> 172.21.15.50:0/2987658523 conn(0x7f965c105240 msgr2=0x7f965c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.539+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 shutdown_connections 2024-05-10T20:55:46.261 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:45.539+0000 7f9663a00700 1 -- 172.21.15.50:0/2987658523 wait complete. 2024-05-10T20:55:46.474 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:46.198+0000 7fc3cf3ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:55:46.475 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:46.346+0000 7fc3cf3ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T20:55:46.737 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:46 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2987658523' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T20:55:46.738 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:46 smithi050 ceph-mon[29546]: mgrmap e9: x(active, since 26s) 2024-05-10T20:55:46.988 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:46.780+0000 7fc3cf3ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T20:55:46.988 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:46.905+0000 7fc3cf3ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T20:55:47.897 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:47 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1876808871' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T20:55:47.907 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:47.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T20:55:47.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T20:55:47.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T20:55:47.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.968+0000 7fdacca00700 1 Processor -- start 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.968+0000 7fdacca00700 1 -- start start 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.968+0000 7fdacca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.969+0000 7fdacca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdac807d770 con 0x7fdac807ee00 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.969+0000 7fdac7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.969+0000 7fdac7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42266/0 (socket says 172.21.15.50:42266) 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.969+0000 7fdac7400700 1 -- 172.21.15.50:0/2711842676 learned_addr learned my addr 172.21.15.50:0/2711842676 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.970+0000 7fdac7400700 1 -- 172.21.15.50:0/2711842676 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac807d8f0 con 0x7fdac807ee00 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.970+0000 7fdac7400700 1 --2- 172.21.15.50:0/2711842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009e80 tx=0x7fdab800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4473253fb5598faa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.970+0000 7fdac6a00700 1 -- 172.21.15.50:0/2711842676 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8017070 con 0x7fdac807ee00 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.970+0000 7fdac6a00700 1 -- 172.21.15.50:0/2711842676 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab800b8c0 con 0x7fdac807ee00 2024-05-10T20:55:47.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.970+0000 7fdac6a00700 1 -- 172.21.15.50:0/2711842676 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8004490 con 0x7fdac807ee00 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.971+0000 7fdacca00700 1 -- 172.21.15.50:0/2711842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 msgr2=0x7fdac807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.971+0000 7fdacca00700 1 --2- 172.21.15.50:0/2711842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009e80 tx=0x7fdab800b330 comp rx=0 tx=0).stop 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.971+0000 7fdacca00700 1 -- 172.21.15.50:0/2711842676 shutdown_connections 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.971+0000 7fdacca00700 1 --2- 172.21.15.50:0/2711842676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac807d230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.971+0000 7fdacca00700 1 -- 172.21.15.50:0/2711842676 >> 172.21.15.50:0/2711842676 conn(0x7fdac8105080 msgr2=0x7fdac8107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.972+0000 7fdacca00700 1 -- 172.21.15.50:0/2711842676 shutdown_connections 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.972+0000 7fdacca00700 1 -- 172.21.15.50:0/2711842676 wait complete. 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.973+0000 7fdacca00700 1 Processor -- start 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.973+0000 7fdacca00700 1 -- start start 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.973+0000 7fdacca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.973+0000 7fdacca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8110040 con 0x7fdac807ee00 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.974+0000 7fdac7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.974+0000 7fdac7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42274/0 (socket says 172.21.15.50:42274) 2024-05-10T20:55:47.915 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.974+0000 7fdac7400700 1 -- 172.21.15.50:0/1876808871 learned_addr learned my addr 172.21.15.50:0/1876808871 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.974+0000 7fdac7400700 1 -- 172.21.15.50:0/1876808871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81a05b0 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.974+0000 7fdac7400700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdab800b7b0 tx=0x7fdab800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.975+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8020020 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.975+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8012430 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.975+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81a0840 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.975+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8017040 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.975+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a3450 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.976+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac8064e20 con 0x7fdac807ee00 2024-05-10T20:55:47.916 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.976+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fdab8023020 con 0x7fdac807ee00 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.977+0000 7fdac4c00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 0x7fdaac040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.977+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdab800f050 con 0x7fdac807ee00 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.977+0000 7fdabea00700 1 -- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 msgr2=0x7fdaac040a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.977+0000 7fdabea00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 0x7fdaac040a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:46.983+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdab802a020 con 0x7fdac807ee00 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.178+0000 7fdabea00700 1 -- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 msgr2=0x7fdaac040a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.178+0000 7fdabea00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 0x7fdaac040a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.273+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdac81a3a50 con 0x7fdac807ee00 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.274+0000 7fdac4c00700 1 -- 172.21.15.50:0/1876808871 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdab802dd60 con 0x7fdac807ee00 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.276+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 msgr2=0x7fdaac040a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.276+0000 7fdacca00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 0x7fdaac040a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.276+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 msgr2=0x7fdac81a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:47.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.276+0000 7fdacca00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdab800b7b0 tx=0x7fdab800bd30 comp rx=0 tx=0).stop 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.276+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 shutdown_connections 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.277+0000 7fdacca00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fdaac03e510 0x7fdaac040a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.277+0000 7fdacca00700 1 --2- 172.21.15.50:0/1876808871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdac807ee00 0x7fdac81a0070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.277+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 >> 172.21.15.50:0/1876808871 conn(0x7fdac8105080 msgr2=0x7fdac81059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.277+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 shutdown_connections 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:47.277+0000 7fdacca00700 1 -- 172.21.15.50:0/1876808871 wait complete. 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-05-10T20:55:47.918 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 9... 2024-05-10T20:55:48.797 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:48.541+0000 7fc3cf3ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T20:55:48.797 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:48.791+0000 7fc3cf3ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T20:55:49.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:49.035+0000 7fc3cf3ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T20:55:49.570 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:49.435+0000 7fc3cf3ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T20:55:49.831 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:49.565+0000 7fc3cf3ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T20:55:49.832 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:49.826+0000 7fc3cf3ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T20:55:50.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:50.095+0000 7fc3cf3ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:55:51.233 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:50.923+0000 7fc3cf3ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T20:55:51.233 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:51.227+0000 7fc3cf3ce200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T20:55:51.922 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:51.821+0000 7fc3cf3ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T20:55:52.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:51.957+0000 7fc3cf3ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T20:55:52.922 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:52.437+0000 7fc3cf3ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T20:55:53.399 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:52.918+0000 7fc3cf3ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T20:55:53.400 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:53.051+0000 7fc3cf3ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T20:55:53.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:53.394+0000 7fc3cf3ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T20:55:53.673 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:53.583+0000 7fc3cf3ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T20:55:54.147 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:53.719+0000 7fc3cf3ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T20:55:54.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:54.142+0000 7fc3cf3ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T20:55:54.423 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:54.274+0000 7fc3cf3ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T20:55:55.173 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:55.078+0000 7fc3cf3ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T20:55:55.672 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:55:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:55:55.386+0000 7fc3cf3ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: Active manager daemon x restarted 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: Activating manager daemon x 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: mgrmap e10: x(active, starting, since 0.0107827s) 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T20:55:55.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:55 smithi050 ceph-mon[29546]: Manager daemon x is now available 2024-05-10T20:55:56.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:55:56.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T20:55:56.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T20:55:56.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:56 smithi050 ceph-mon[29546]: mgrmap e11: x(active, since 1.00912s) 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.624+0000 7fe09dc00700 1 Processor -- start 2024-05-10T20:55:57.082 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.624+0000 7fe09dc00700 1 -- start start 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.625+0000 7fe09dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.625+0000 7fe09dc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe09810a5d0 con 0x7fe098109bd0 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.625+0000 7fe096a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.625+0000 7fe096a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42290/0 (socket says 172.21.15.50:42290) 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.625+0000 7fe096a00700 1 -- 172.21.15.50:0/1824592338 learned_addr learned my addr 172.21.15.50:0/1824592338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.626+0000 7fe096a00700 1 -- 172.21.15.50:0/1824592338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09810ada0 con 0x7fe098109bd0 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.626+0000 7fe096a00700 1 --2- 172.21.15.50:0/1824592338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0880098a0 tx=0x7fe088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5295ac43766bb152 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.627+0000 7fe095600700 1 -- 172.21.15.50:0/1824592338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe088004160 con 0x7fe098109bd0 2024-05-10T20:55:57.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.627+0000 7fe095600700 1 -- 172.21.15.50:0/1824592338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe088004300 con 0x7fe098109bd0 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.627+0000 7fe095600700 1 -- 172.21.15.50:0/1824592338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe088011a20 con 0x7fe098109bd0 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.627+0000 7fe09dc00700 1 -- 172.21.15.50:0/1824592338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 msgr2=0x7fe098109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.627+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1824592338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0880098a0 tx=0x7fe088009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.628+0000 7fe09dc00700 1 -- 172.21.15.50:0/1824592338 shutdown_connections 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.628+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1824592338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe098109fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.628+0000 7fe09dc00700 1 -- 172.21.15.50:0/1824592338 >> 172.21.15.50:0/1824592338 conn(0x7fe098105210 msgr2=0x7fe0981076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.628+0000 7fe09dc00700 1 -- 172.21.15.50:0/1824592338 shutdown_connections 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.628+0000 7fe09dc00700 1 -- 172.21.15.50:0/1824592338 wait complete. 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.629+0000 7fe09dc00700 1 Processor -- start 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.629+0000 7fe09dc00700 1 -- start start 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe09dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe09dc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe09810e150 con 0x7fe098109bd0 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe096a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:57.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe096a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42292/0 (socket says 172.21.15.50:42292) 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe096a00700 1 -- 172.21.15.50:0/1826659177 learned_addr learned my addr 172.21.15.50:0/1826659177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.630+0000 7fe096a00700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0981a0750 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe096a00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe088011a20 tx=0x7fe088011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0880039c0 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe088003b60 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe088019700 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0981a09e0 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.631+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a35f0 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fe08801a430 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe08f400700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe07c0410f0 con 0x7fe07c03e4d0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe088053dc0 con 0x7fe098109bd0 2024-05-10T20:55:57.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.633+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.834+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:48.834+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:49.235+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:49.235+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:50.036+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:50.036+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:51.638+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:51.638+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:54.842+0000 7fe096000700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/3321516664 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:54.842+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:55.396+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fe08801ab00 con 0x7fe098109bd0 2024-05-10T20:55:57.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:55.396+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 msgr2=0x7fe07c0409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:55.396+0000 7fe08f400700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.395+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fe08802a2b0 con 0x7fe098109bd0 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.396+0000 7fe08f400700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 0x7fe07c043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.396+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe08801be50 con 0x7fe088053950 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.397+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 0x7fe07c043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.399+0000 7fe096000700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 0x7fe07c043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe084003a10 tx=0x7fe084009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.400+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe08801be50 con 0x7fe088053950 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.411+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe098064e20 con 0x7fe088053950 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.411+0000 7fe08f400700 1 -- 172.21.15.50:0/1826659177 <== mgr.14152 v2:172.21.15.50:6800/628586267 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fe098064e20 con 0x7fe088053950 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 msgr2=0x7fe07c043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 0x7fe07c043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe084003a10 tx=0x7fe084009270 comp rx=0 tx=0).stop 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 msgr2=0x7fe0981a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:57.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe088011a20 tx=0x7fe088011b00 comp rx=0 tx=0).stop 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 shutdown_connections 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe088053950 0x7fe07c043fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe098109bd0 0x7fe0981a0210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 --2- 172.21.15.50:0/1826659177 >> [v2:172.21.15.50:6800/3321516664,v1:172.21.15.50:6801/3321516664] conn(0x7fe07c03e4d0 0x7fe07c0409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.412+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 >> 172.21.15.50:0/1826659177 conn(0x7fe098105210 msgr2=0x7fe09807d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.413+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 shutdown_connections 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:56.413+0000 7fe09dc00700 1 -- 172.21.15.50:0/1826659177 wait complete. 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 9 is available 2024-05-10T20:55:57.095 INFO:teuthology.orchestra.run.smithi050.stdout:Generating a dashboard self-signed certificate... 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: [10/May/2024:20:55:56] ENGINE Bus STARTING 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: [10/May/2024:20:55:56] ENGINE Serving on https://172.21.15.50:7150 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: [10/May/2024:20:55:56] ENGINE Serving on http://172.21.15.50:8765 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: [10/May/2024:20:55:56] ENGINE Bus STARTED 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T20:55:57.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:57 smithi050 ceph-mon[29546]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T20:55:58.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:58 smithi050 ceph-mon[29546]: mgrmap e12: x(active, since 2s) 2024-05-10T20:55:58.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:55:59.559 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T20:55:59.559 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.400+0000 7fae30000700 1 Processor -- start 2024-05-10T20:55:59.559 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.400+0000 7fae30000700 1 -- start start 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.400+0000 7fae30000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.400+0000 7fae30000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae2807ce70 con 0x7fae2810a6f0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.401+0000 7fae2d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.401+0000 7fae2d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49532/0 (socket says 172.21.15.50:49532) 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.401+0000 7fae2d600700 1 -- 172.21.15.50:0/3636638607 learned_addr learned my addr 172.21.15.50:0/3636638607 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.402+0000 7fae2d600700 1 -- 172.21.15.50:0/3636638607 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2810d050 con 0x7fae2810a6f0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.402+0000 7fae2d600700 1 --2- 172.21.15.50:0/3636638607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0098a0 tx=0x7fae1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24e970bc205918f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.403+0000 7fae27e00700 1 -- 172.21.15.50:0/3636638607 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae1c004160 con 0x7fae2810a6f0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.403+0000 7fae27e00700 1 -- 172.21.15.50:0/3636638607 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1c004300 con 0x7fae2810a6f0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.403+0000 7fae27e00700 1 -- 172.21.15.50:0/3636638607 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae1c011a20 con 0x7fae2810a6f0 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 -- 172.21.15.50:0/3636638607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 msgr2=0x7fae2810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 --2- 172.21.15.50:0/3636638607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0098a0 tx=0x7fae1c009bb0 comp rx=0 tx=0).stop 2024-05-10T20:55:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 -- 172.21.15.50:0/3636638607 shutdown_connections 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 --2- 172.21.15.50:0/3636638607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae2810cb10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 -- 172.21.15.50:0/3636638607 >> 172.21.15.50:0/3636638607 conn(0x7fae281041d0 msgr2=0x7fae28106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 -- 172.21.15.50:0/3636638607 shutdown_connections 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.404+0000 7fae30000700 1 -- 172.21.15.50:0/3636638607 wait complete. 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.405+0000 7fae30000700 1 Processor -- start 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae30000700 1 -- start start 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae30000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae30000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810e050 con 0x7fae2810a6f0 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae2d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae2d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49536/0 (socket says 172.21.15.50:49536) 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.406+0000 7fae2d600700 1 -- 172.21.15.50:0/1799422027 learned_addr learned my addr 172.21.15.50:0/1799422027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.407+0000 7fae2d600700 1 -- 172.21.15.50:0/1799422027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae281a0740 con 0x7fae2810a6f0 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.407+0000 7fae2d600700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae1c004660 tx=0x7fae1c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:59.561 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.408+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae1c003a60 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.408+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1c01b8f0 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.408+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae281a09d0 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.408+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae1c023890 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.408+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a35e0 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.410+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae28064e20 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.410+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fae1c02c3f0 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.411+0000 7fae26000700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 0x7fae100409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.411+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae1c054120 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.411+0000 7fae2cc00700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 0x7fae100409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.412+0000 7fae2cc00700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 0x7fae100409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae14006fd0 tx=0x7fae14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.417+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae1c01f030 con 0x7fae2810a6f0 2024-05-10T20:55:59.562 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.665+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fae2802e8f0 con 0x7fae1003e4d0 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.831+0000 7fae26000700 1 -- 172.21.15.50:0/1799422027 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fae2802e8f0 con 0x7fae1003e4d0 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 msgr2=0x7fae100409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 0x7fae100409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae14006fd0 tx=0x7fae14006e40 comp rx=0 tx=0).stop 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 msgr2=0x7fae281a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae1c004660 tx=0x7fae1c011aa0 comp rx=0 tx=0).stop 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 shutdown_connections 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fae1003e4d0 0x7fae100409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 --2- 172.21.15.50:0/1799422027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae2810a6f0 0x7fae281a0200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 >> 172.21.15.50:0/1799422027 conn(0x7fae281041d0 msgr2=0x7fae28108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 shutdown_connections 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:55:58.835+0000 7fae30000700 1 -- 172.21.15.50:0/1799422027 wait complete. 2024-05-10T20:55:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial admin user... 2024-05-10T20:55:59.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:59 smithi050 ceph-mon[29546]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:55:59.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:55:59.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:55:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:01.371 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:01.371 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:01 smithi050 ceph-mon[29546]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:01.372 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:01.381 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iU.UvMXWY/gmwhAZ0ClhvO6.Ik2YUSKF/t0zBaYbEQO/1xncxklFm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715374560, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T20:56:01.381 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.208+0000 7fac44a00700 1 Processor -- start 2024-05-10T20:56:01.382 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.208+0000 7fac44a00700 1 -- start start 2024-05-10T20:56:01.382 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.208+0000 7fac44a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:01.382 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.208+0000 7fac44a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac4007d770 con 0x7fac4007ee00 2024-05-10T20:56:01.390 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.209+0000 7fac3de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:01.390 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.209+0000 7fac3de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49540/0 (socket says 172.21.15.50:49540) 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.209+0000 7fac3de00700 1 -- 172.21.15.50:0/2167248609 learned_addr learned my addr 172.21.15.50:0/2167248609 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.209+0000 7fac3de00700 1 -- 172.21.15.50:0/2167248609 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4007d8f0 con 0x7fac4007ee00 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.210+0000 7fac3de00700 1 --2- 172.21.15.50:0/2167248609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac300098a0 tx=0x7fac30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee74428a6babe25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.210+0000 7fac3ca00700 1 -- 172.21.15.50:0/2167248609 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30004160 con 0x7fac4007ee00 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.210+0000 7fac3ca00700 1 -- 172.21.15.50:0/2167248609 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac30004300 con 0x7fac4007ee00 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.210+0000 7fac3ca00700 1 -- 172.21.15.50:0/2167248609 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30011a20 con 0x7fac4007ee00 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 -- 172.21.15.50:0/2167248609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 msgr2=0x7fac4007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 --2- 172.21.15.50:0/2167248609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac300098a0 tx=0x7fac30009bb0 comp rx=0 tx=0).stop 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 -- 172.21.15.50:0/2167248609 shutdown_connections 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 --2- 172.21.15.50:0/2167248609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac4007d230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 -- 172.21.15.50:0/2167248609 >> 172.21.15.50:0/2167248609 conn(0x7fac401052a0 msgr2=0x7fac40107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 -- 172.21.15.50:0/2167248609 shutdown_connections 2024-05-10T20:56:01.391 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.211+0000 7fac44a00700 1 -- 172.21.15.50:0/2167248609 wait complete. 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.212+0000 7fac44a00700 1 Processor -- start 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac44a00700 1 -- start start 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac44a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac44a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac40110240 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac3de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac3de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49554/0 (socket says 172.21.15.50:49554) 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.213+0000 7fac3de00700 1 -- 172.21.15.50:0/361902640 learned_addr learned my addr 172.21.15.50:0/361902640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac3de00700 1 -- 172.21.15.50:0/361902640 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac401a0710 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac3de00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac30009520 tx=0x7fac30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac300039e0 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3001b8f0 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac401a09a0 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.214+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac300197d0 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.215+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac401a35b0 con 0x7fac4007ee00 2024-05-10T20:56:01.392 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.216+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac40064e20 con 0x7fac4007ee00 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.216+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fac30019ab0 con 0x7fac4007ee00 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.216+0000 7fac36a00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 0x7fac240409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.216+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac300541f0 con 0x7fac4007ee00 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.217+0000 7fac3d400700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 0x7fac240409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.218+0000 7fac3d400700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 0x7fac240409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac2c006fd0 tx=0x7fac2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.222+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac3001f030 con 0x7fac4007ee00 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.477+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fac401a3780 con 0x7fac2403e4c0 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.743+0000 7fac36a00700 1 -- 172.21.15.50:0/361902640 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fac401a3780 con 0x7fac2403e4c0 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 msgr2=0x7fac240409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 0x7fac240409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac2c006fd0 tx=0x7fac2c006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 msgr2=0x7fac401a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac30009520 tx=0x7fac30011aa0 comp rx=0 tx=0).stop 2024-05-10T20:56:01.393 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 shutdown_connections 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fac2403e4c0 0x7fac240409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 --2- 172.21.15.50:0/361902640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac4007ee00 0x7fac401a01d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.745+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 >> 172.21.15.50:0/361902640 conn(0x7fac401052a0 msgr2=0x7fac40105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.746+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 shutdown_connections 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:00.746+0000 7fac44a00700 1 -- 172.21.15.50:0/361902640 wait complete. 2024-05-10T20:56:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:Fetching dashboard port number... 2024-05-10T20:56:03.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:03 smithi050 ceph-mon[29546]: mgrmap e13: x(active, since 6s) 2024-05-10T20:56:03.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:03 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1050781203' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T20:56:03.785 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.337+0000 7f3ac1e00700 1 Processor -- start 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.338+0000 7f3ac1e00700 1 -- start start 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.338+0000 7f3ac1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.338+0000 7f3ac1e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10a610 con 0x7f3abc109c10 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.339+0000 7f3abaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:03.792 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.339+0000 7f3abaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49560/0 (socket says 172.21.15.50:49560) 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.339+0000 7f3abaa00700 1 -- 172.21.15.50:0/2361222770 learned_addr learned my addr 172.21.15.50:0/2361222770 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.339+0000 7f3abaa00700 1 -- 172.21.15.50:0/2361222770 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc10ade0 con 0x7f3abc109c10 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.340+0000 7f3abaa00700 1 --2- 172.21.15.50:0/2361222770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098a0 tx=0x7f3aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1e9e62cbe8dd4d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.340+0000 7f3ab9600700 1 -- 172.21.15.50:0/2361222770 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac004160 con 0x7f3abc109c10 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.340+0000 7f3ab9600700 1 -- 172.21.15.50:0/2361222770 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac004300 con 0x7f3abc109c10 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.340+0000 7f3ab9600700 1 -- 172.21.15.50:0/2361222770 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac011a20 con 0x7f3abc109c10 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 -- 172.21.15.50:0/2361222770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 msgr2=0x7f3abc10a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/2361222770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098a0 tx=0x7f3aac009bb0 comp rx=0 tx=0).stop 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 -- 172.21.15.50:0/2361222770 shutdown_connections 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/2361222770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc10a020 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 -- 172.21.15.50:0/2361222770 >> 172.21.15.50:0/2361222770 conn(0x7f3abc1052d0 msgr2=0x7f3abc107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 -- 172.21.15.50:0/2361222770 shutdown_connections 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.341+0000 7f3ac1e00700 1 -- 172.21.15.50:0/2361222770 wait complete. 2024-05-10T20:56:03.793 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.343+0000 7f3ac1e00700 1 Processor -- start 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.343+0000 7f3ac1e00700 1 -- start start 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.343+0000 7f3ac1e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.343+0000 7f3ac1e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10e190 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.344+0000 7f3abaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.344+0000 7f3abaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49564/0 (socket says 172.21.15.50:49564) 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.344+0000 7f3abaa00700 1 -- 172.21.15.50:0/1050781203 learned_addr learned my addr 172.21.15.50:0/1050781203 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.344+0000 7f3abaa00700 1 -- 172.21.15.50:0/1050781203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc07f830 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.344+0000 7f3abaa00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009130 tx=0x7f3aac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.345+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac003aa0 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.345+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac01a8e0 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.345+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc07da80 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.345+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac0248f0 con 0x7f3abc109c10 2024-05-10T20:56:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.345+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc07df60 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.346+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc064e20 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.347+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3aac022020 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.347+0000 7f3ac0a00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 0x7f3aa0040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.347+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3aac053ad0 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.347+0000 7f3aba000700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 0x7f3aa0040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.348+0000 7f3aba000700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 0x7f3aa0040a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006fd0 tx=0x7f3aa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.353+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3aac00f2e0 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.589+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3abc07e610 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.590+0000 7f3ac0a00700 1 -- 172.21.15.50:0/1050781203 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3aac024ba0 con 0x7f3abc109c10 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.593+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 msgr2=0x7f3aa0040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.593+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 0x7f3aa0040a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006fd0 tx=0x7f3aa8006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 msgr2=0x7f3abc07f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009130 tx=0x7f3aac011aa0 comp rx=0 tx=0).stop 2024-05-10T20:56:03.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 shutdown_connections 2024-05-10T20:56:03.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3aa003e560 0x7f3aa0040a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:03.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 --2- 172.21.15.50:0/1050781203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3abc109c10 0x7f3abc07f2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:03.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 >> 172.21.15.50:0/1050781203 conn(0x7f3abc1052d0 msgr2=0x7f3abc106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:03.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 shutdown_connections 2024-05-10T20:56:03.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:02.594+0000 7f3ac1e00700 1 -- 172.21.15.50:0/1050781203 wait complete. 2024-05-10T20:56:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T20:56:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-10T20:56:03.855 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T20:56:03.856 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-10T20:56:03.856 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-10T20:56:03.856 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph Dashboard is now available at: 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout: URL: https://smithi050.front.sepia.ceph.com:8443/ 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout: User: admin 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout: Password: w9wsr1houn 2024-05-10T20:56:03.864 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:03.865 INFO:teuthology.orchestra.run.smithi050.stdout:Saving cluster configuration to /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config directory 2024-05-10T20:56:03.867 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling autotune for osd_memory_target 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.577+0000 7ff0f5c00700 1 Processor -- start 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.577+0000 7ff0f5c00700 1 -- start start 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.578+0000 7ff0f5c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.578+0000 7ff0f5c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010a600 con 0x7ff0f0109c00 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.578+0000 7ff0eea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:05.587 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.578+0000 7ff0eea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35568/0 (socket says 172.21.15.50:35568) 2024-05-10T20:56:05.588 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.578+0000 7ff0eea00700 1 -- 172.21.15.50:0/1156557050 learned_addr learned my addr 172.21.15.50:0/1156557050 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:05.588 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.579+0000 7ff0eea00700 1 -- 172.21.15.50:0/1156557050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f010add0 con 0x7ff0f0109c00 2024-05-10T20:56:05.588 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.579+0000 7ff0eea00700 1 --2- 172.21.15.50:0/1156557050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00098a0 tx=0x7ff0e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115c501641aafd77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.580+0000 7ff0ed600700 1 -- 172.21.15.50:0/1156557050 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e0004160 con 0x7ff0f0109c00 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.580+0000 7ff0ed600700 1 -- 172.21.15.50:0/1156557050 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e0004300 con 0x7ff0f0109c00 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.580+0000 7ff0ed600700 1 -- 172.21.15.50:0/1156557050 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e0011a20 con 0x7ff0f0109c00 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.580+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1156557050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 msgr2=0x7ff0f010a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.580+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1156557050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00098a0 tx=0x7ff0e0009bb0 comp rx=0 tx=0).stop 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.581+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1156557050 shutdown_connections 2024-05-10T20:56:05.589 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.581+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1156557050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f010a010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.581+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1156557050 >> 172.21.15.50:0/1156557050 conn(0x7ff0f0105380 msgr2=0x7ff0f01077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.581+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1156557050 shutdown_connections 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.581+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1156557050 wait complete. 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.582+0000 7ff0f5c00700 1 Processor -- start 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.582+0000 7ff0f5c00700 1 -- start start 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0f5c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0f5c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010e180 con 0x7ff0f0109c00 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0eea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0eea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35574/0 (socket says 172.21.15.50:35574) 2024-05-10T20:56:05.591 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0eea00700 1 -- 172.21.15.50:0/1944118743 learned_addr learned my addr 172.21.15.50:0/1944118743 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.583+0000 7ff0eea00700 1 -- 172.21.15.50:0/1944118743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f019e5a0 con 0x7ff0f0109c00 2024-05-10T20:56:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0eea00700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0011a20 tx=0x7ff0e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e00039c0 con 0x7ff0f0109c00 2024-05-10T20:56:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f019e830 con 0x7ff0f0109c00 2024-05-10T20:56:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e0003b60 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e0019700 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.584+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f007d960 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.585+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0064e20 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.585+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff0e0019a00 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.586+0000 7ff0e7400700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 0x7ff0d4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.586+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff0e0054090 con 0x7ff0f0109c00 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.586+0000 7ff0ee000700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 0x7ff0d4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.587+0000 7ff0ee000700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 0x7ff0d4040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc006fd0 tx=0x7ff0dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:05.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.592+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0e000b3e0 con 0x7ff0f0109c00 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.826+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff0f007dfd0 con 0x7ff0f0109c00 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.827+0000 7ff0e7400700 1 -- 172.21.15.50:0/1944118743 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff0e0023e00 con 0x7ff0f0109c00 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 msgr2=0x7ff0d4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 0x7ff0d4040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc006fd0 tx=0x7ff0dc006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 msgr2=0x7ff0f019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0011a20 tx=0x7ff0e0011b00 comp rx=0 tx=0).stop 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 shutdown_connections 2024-05-10T20:56:05.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0d403e510 0x7ff0d4040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:05.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 --2- 172.21.15.50:0/1944118743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f0109c00 0x7ff0f019e060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:05.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.830+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 >> 172.21.15.50:0/1944118743 conn(0x7ff0f0105380 msgr2=0x7ff0f0106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:05.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.831+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 shutdown_connections 2024-05-10T20:56:05.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:04.831+0000 7ff0f5c00700 1 -- 172.21.15.50:0/1944118743 wait complete. 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.276+0000 7f8283200700 1 Processor -- start 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.277+0000 7f8283200700 1 -- start start 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.277+0000 7f8283200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.277+0000 7f8283200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f827c07d840 con 0x7f827c07ee40 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.278+0000 7f8281e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.278+0000 7f8281e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35588/0 (socket says 172.21.15.50:35588) 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.278+0000 7f8281e00700 1 -- 172.21.15.50:0/2067442108 learned_addr learned my addr 172.21.15.50:0/2067442108 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.278+0000 7f8281e00700 1 -- 172.21.15.50:0/2067442108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c07d9c0 con 0x7f827c07ee40 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.279+0000 7f8281e00700 1 --2- 172.21.15.50:0/2067442108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82700098a0 tx=0x7f8270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2d498087b45e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:07.358 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.279+0000 7f8280a00700 1 -- 172.21.15.50:0/2067442108 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8270004160 con 0x7f827c07ee40 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.279+0000 7f8280a00700 1 -- 172.21.15.50:0/2067442108 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8270004300 con 0x7f827c07ee40 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.279+0000 7f8280a00700 1 -- 172.21.15.50:0/2067442108 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8270011a20 con 0x7f827c07ee40 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 -- 172.21.15.50:0/2067442108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 msgr2=0x7f827c07d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 --2- 172.21.15.50:0/2067442108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82700098a0 tx=0x7f8270009bb0 comp rx=0 tx=0).stop 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 -- 172.21.15.50:0/2067442108 shutdown_connections 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 --2- 172.21.15.50:0/2067442108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c07d270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 -- 172.21.15.50:0/2067442108 >> 172.21.15.50:0/2067442108 conn(0x7f827c105080 msgr2=0x7f827c107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 -- 172.21.15.50:0/2067442108 shutdown_connections 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.280+0000 7f8283200700 1 -- 172.21.15.50:0/2067442108 wait complete. 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.281+0000 7f8283200700 1 Processor -- start 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8283200700 1 -- start start 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8283200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:07.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8283200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f827c110070 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8281e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8281e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35590/0 (socket says 172.21.15.50:35590) 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.282+0000 7f8281e00700 1 -- 172.21.15.50:0/1719788347 learned_addr learned my addr 172.21.15.50:0/1719788347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.283+0000 7f8281e00700 1 -- 172.21.15.50:0/1719788347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c1a05e0 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.283+0000 7f8281e00700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8270009520 tx=0x7f8270011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.283+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82700039e0 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.283+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827001b8f0 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.283+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c1a0870 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.284+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82700197d0 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.284+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c1a3480 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.285+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f827001b440 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.285+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c064e20 con 0x7f827c07ee40 2024-05-10T20:56:07.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.286+0000 7f8276a00700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 0x7f8264040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.286+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82700540a0 con 0x7f827c07ee40 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.286+0000 7f8281400700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 0x7f8264040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.287+0000 7f8281400700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 0x7f8264040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f826c006fd0 tx=0x7f826c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.292+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f827001f030 con 0x7f827c07ee40 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.650+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f827c1a3650 con 0x7f827c07ee40 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.655+0000 7f8276a00700 1 -- 172.21.15.50:0/1719788347 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f827000f2e0 con 0x7f827c07ee40 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.658+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 msgr2=0x7f8264040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.658+0000 7f8283200700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 0x7f8264040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f826c006fd0 tx=0x7f826c006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.658+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 msgr2=0x7f827c1a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.658+0000 7f8283200700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8270009520 tx=0x7f8270011aa0 comp rx=0 tx=0).stop 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 shutdown_connections 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f826403e560 0x7f8264040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 --2- 172.21.15.50:0/1719788347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f827c07ee40 0x7f827c1a00a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:07.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 >> 172.21.15.50:0/1719788347 conn(0x7f827c105080 msgr2=0x7f827c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 shutdown_connections 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-10T20:56:06.659+0000 7f8283200700 1 -- 172.21.15.50:0/1719788347 wait complete. 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.362 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: ceph telemetry on 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout:For more information see: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:07.363 INFO:teuthology.orchestra.run.smithi050.stdout:Bootstrap complete. 2024-05-10T20:56:07.455 INFO:tasks.cephadm:Fetching config... 2024-05-10T20:56:07.456 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:56:07.456 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T20:56:07.501 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T20:56:07.501 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:56:07.502 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T20:56:07.583 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T20:56:07.583 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:56:07.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T20:56:07.708 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T20:56:07.708 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:56:07.708 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T20:56:07.764 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T20:56:07.764 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T20:56:07.878 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:07.910 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:07 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1719788347' entity='client.admin' 2024-05-10T20:56:07.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T20:56:07.966 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:07.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T20:56:08.019 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBP5ebcbRTKyZcRniUjZFSbzXSN8bMSEziqa7gUoVgiHOQ/FGriy1L6czln8VN034RO3s0tl+GS/5Voy0MRlwh9XHGVXKix/wcohVeKHj9cEtSCs7QwAmAJdI2yeKkasX2hVDjnqwYAF1jiI2RtMDgUHxZ3Whj0kOmdKbQ9GuavEsuV/Y8WoWRWcEqwEu6pSLbVl5ph5D8iGc3NNuXENb/Kvi9H9+G5K4CgC8hA06jmCrQUH7oy5Vy7ddsUTz/duoEXlXxmDV4P3quC8nBdJp98ExI0ZMYRvSniaSc6zsC4pcCLGUZvAhCvTk+m5QiiUkiH8Wsu1SEHUyoJ0wJKIVnXnNi83+nhE9RKWdKFVvGHnJHrFZ6axYECxmKPPLk3g7z5XoBy4F8xQzqt6YEMOaiHSD4YWCJPy2+NANidPR1MJhXI3jrIbeeVlFr60HOcoqf2WQ2s8ZQXXr7H4jjaa88A0g98su+8uVFg45i6BlaO28lN/5uUR5ODIUTn5wgjCU= ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:08.030 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T20:56:08.486 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:10.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3091021593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 msgr2=0x7f6a50075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:10.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3091021593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a50075f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009910 tx=0x7f6a40009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:10.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3091021593 shutdown_connections 2024-05-10T20:56:10.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3091021593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a50075f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3091021593 >> 172.21.15.50:0/3091021593 conn(0x7f6a500ff050 msgr2=0x7f6a50101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3091021593 shutdown_connections 2024-05-10T20:56:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.750+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3091021593 wait complete. 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.752+0000 7f6a4fe00700 1 Processor -- start 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.752+0000 7f6a4fe00700 1 -- start start 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.752+0000 7f6a4fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.752+0000 7f6a4fe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5010bcd0 con 0x7f6a50075b50 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.753+0000 7f6a4ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.753+0000 7f6a4ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35612/0 (socket says 172.21.15.50:35612) 2024-05-10T20:56:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.753+0000 7f6a4ea00700 1 -- 172.21.15.50:0/3009338869 learned_addr learned my addr 172.21.15.50:0/3009338869 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.753+0000 7f6a4ea00700 1 -- 172.21.15.50:0/3009338869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5019bdb0 con 0x7f6a50075b50 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.753+0000 7f6a4ea00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a40006010 tx=0x7f6a40005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.754+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a40026070 con 0x7f6a50075b50 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.754+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a400044a0 con 0x7f6a50075b50 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.754+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a400039d0 con 0x7f6a50075b50 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.754+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5019c040 con 0x7f6a50075b50 2024-05-10T20:56:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.754+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5019ec50 con 0x7f6a50075b50 2024-05-10T20:56:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.755+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6a40029070 con 0x7f6a50075b50 2024-05-10T20:56:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.755+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5010c450 con 0x7f6a50075b50 2024-05-10T20:56:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.756+0000 7f6a47400700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 0x7f6a34040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.756+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a4001e070 con 0x7f6a50075b50 2024-05-10T20:56:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.756+0000 7f6a47e00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 0x7f6a34040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.757+0000 7f6a47e00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 0x7f6a34040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6a38006fd0 tx=0x7f6a38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:10.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:10.762+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a40013070 con 0x7f6a50075b50 2024-05-10T20:56:11.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.002+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6a5002e5f0 con 0x7f6a50075b50 2024-05-10T20:56:11.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.006+0000 7f6a47400700 1 -- 172.21.15.50:0/3009338869 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6a400300c0 con 0x7f6a50075b50 2024-05-10T20:56:11.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.009+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 msgr2=0x7f6a34040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.009+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 0x7f6a34040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6a38006fd0 tx=0x7f6a38006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.009+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 msgr2=0x7f6a5019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.009+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a40006010 tx=0x7f6a40005ad0 comp rx=0 tx=0).stop 2024-05-10T20:56:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 shutdown_connections 2024-05-10T20:56:11.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a3403e570 0x7f6a34040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:11.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 --2- 172.21.15.50:0/3009338869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a50075b50 0x7f6a5019b870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:11.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 >> 172.21.15.50:0/3009338869 conn(0x7f6a500ff050 msgr2=0x7f6a500ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:11.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 shutdown_connections 2024-05-10T20:56:11.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:11.010+0000 7f6a4fe00700 1 -- 172.21.15.50:0/3009338869 wait complete. 2024-05-10T20:56:11.823 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T20:56:11.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T20:56:12.267 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:12.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3009338869' entity='client.admin' 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 -- 172.21.15.50:0/502316956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 msgr2=0x7fc308105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 --2- 172.21.15.50:0/502316956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc308105500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009910 tx=0x7fc2f8009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:14.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 -- 172.21.15.50:0/502316956 shutdown_connections 2024-05-10T20:56:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 --2- 172.21.15.50:0/502316956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc308105500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 -- 172.21.15.50:0/502316956 >> 172.21.15.50:0/502316956 conn(0x7fc3080ff050 msgr2=0x7fc308101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.386+0000 7fc307e00700 1 -- 172.21.15.50:0/502316956 shutdown_connections 2024-05-10T20:56:14.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.387+0000 7fc307e00700 1 -- 172.21.15.50:0/502316956 wait complete. 2024-05-10T20:56:14.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.388+0000 7fc307e00700 1 Processor -- start 2024-05-10T20:56:14.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.388+0000 7fc307e00700 1 -- start start 2024-05-10T20:56:14.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.389+0000 7fc307e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:14.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.389+0000 7fc307e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc308109ad0 con 0x7fc3081050f0 2024-05-10T20:56:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.389+0000 7fc306a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.389+0000 7fc306a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44156/0 (socket says 172.21.15.50:44156) 2024-05-10T20:56:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.389+0000 7fc306a00700 1 -- 172.21.15.50:0/3873918391 learned_addr learned my addr 172.21.15.50:0/3873918391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:14.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.390+0000 7fc306a00700 1 -- 172.21.15.50:0/3873918391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a0210 con 0x7fc3081050f0 2024-05-10T20:56:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.390+0000 7fc306a00700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006010 tx=0x7fc2f8004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.391+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8026070 con 0x7fc3081050f0 2024-05-10T20:56:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.391+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f800bc00 con 0x7fc3081050f0 2024-05-10T20:56:14.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.391+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8003970 con 0x7fc3081050f0 2024-05-10T20:56:14.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.391+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a04a0 con 0x7fc3081050f0 2024-05-10T20:56:14.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.391+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a30b0 con 0x7fc3081050f0 2024-05-10T20:56:14.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.392+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30810a250 con 0x7fc3081050f0 2024-05-10T20:56:14.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.392+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc2f8029070 con 0x7fc3081050f0 2024-05-10T20:56:14.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.393+0000 7fc2ff400700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 0x7fc2ec040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:14.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.393+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc2f801e070 con 0x7fc3081050f0 2024-05-10T20:56:14.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.394+0000 7fc306000700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 0x7fc2ec040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:14.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.394+0000 7fc306000700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 0x7fc2ec040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:14.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.399+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2f8013070 con 0x7fc3081050f0 2024-05-10T20:56:14.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.643+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc308063c00 con 0x7fc2ec03e570 2024-05-10T20:56:14.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.649+0000 7fc2ff400700 1 -- 172.21.15.50:0/3873918391 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc308063c00 con 0x7fc2ec03e570 2024-05-10T20:56:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 msgr2=0x7fc2ec040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:14.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 0x7fc2ec040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 msgr2=0x7fc30819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006010 tx=0x7fc2f8004fa0 comp rx=0 tx=0).stop 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 shutdown_connections 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2ec03e570 0x7fc2ec040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 --2- 172.21.15.50:0/3873918391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3081050f0 0x7fc30819fcd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.652+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 >> 172.21.15.50:0/3873918391 conn(0x7fc3080ff050 msgr2=0x7fc308101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.653+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 shutdown_connections 2024-05-10T20:56:14.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:14.653+0000 7fc307e00700 1 -- 172.21.15.50:0/3873918391 wait complete. 2024-05-10T20:56:15.495 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-05-10T20:56:15.495 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:56:15.495 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-05-10T20:56:15.513 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:56:15.513 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:15.571 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-05-10T20:56:15.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch host add smithi092 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:15.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:15 smithi050 ceph-mon[29546]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-10T20:56:15.985 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:16.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:16 smithi050 ceph-mon[29546]: Updating smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:16.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:16 smithi050 ceph-mon[29546]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:17.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:17 smithi050 ceph-mon[29546]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:17.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:17 smithi050 ceph-mon[29546]: Updating smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:56:17.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:17 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:17.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:17 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:17.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:17 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 -- 172.21.15.50:0/2030303464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 msgr2=0x7f0ee0109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 --2- 172.21.15.50:0/2030303464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0109db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009910 tx=0x7f0ed0009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:18.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 -- 172.21.15.50:0/2030303464 shutdown_connections 2024-05-10T20:56:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 --2- 172.21.15.50:0/2030303464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0109db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 -- 172.21.15.50:0/2030303464 >> 172.21.15.50:0/2030303464 conn(0x7f0ee00ff4b0 msgr2=0x7f0ee0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 -- 172.21.15.50:0/2030303464 shutdown_connections 2024-05-10T20:56:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.111+0000 7f0edfe00700 1 -- 172.21.15.50:0/2030303464 wait complete. 2024-05-10T20:56:18.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.112+0000 7f0edfe00700 1 Processor -- start 2024-05-10T20:56:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.112+0000 7f0edfe00700 1 -- start start 2024-05-10T20:56:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edfe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:18.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edfe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee010def0 con 0x7f0ee01099a0 2024-05-10T20:56:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44164/0 (socket says 172.21.15.50:44164) 2024-05-10T20:56:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edea00700 1 -- 172.21.15.50:0/1357486708 learned_addr learned my addr 172.21.15.50:0/1357486708 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.113+0000 7f0edea00700 1 -- 172.21.15.50:0/1357486708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee0077910 con 0x7f0ee01099a0 2024-05-10T20:56:18.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0edea00700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00092e0 tx=0x7f0ed0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed0026070 con 0x7f0ee01099a0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed000b6f0 con 0x7f0ee01099a0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed00037f0 con 0x7f0ee01099a0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee0076000 con 0x7f0ee01099a0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.114+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee00764c0 con 0x7f0ee01099a0 2024-05-10T20:56:18.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.116+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee01050b0 con 0x7f0ee01099a0 2024-05-10T20:56:18.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.116+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0ed0029070 con 0x7f0ee01099a0 2024-05-10T20:56:18.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.116+0000 7f0ed7400700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 0x7f0ec4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:18.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.116+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ed001e070 con 0x7f0ee01099a0 2024-05-10T20:56:18.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.117+0000 7f0ede000700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 0x7f0ec4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:18.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.117+0000 7f0ede000700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 0x7f0ec4040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006fd0 tx=0x7f0ecc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:18.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.123+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ed0013070 con 0x7f0ee01099a0 2024-05-10T20:56:18.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:18.369+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) -- 0x7f0ee0077090 con 0x7f0ec403e570 2024-05-10T20:56:18.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:18 smithi050 ceph-mon[29546]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:19.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:19 smithi050 ceph-mon[29546]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:20.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:20 smithi050 ceph-mon[29546]: Deploying cephadm binary to smithi092 2024-05-10T20:56:20.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:20 smithi050 ceph-mon[29546]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:22.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:22 smithi050 ceph-mon[29546]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:23.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.191+0000 7f0ed7400700 1 -- 172.21.15.50:0/1357486708 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0ee0077090 con 0x7f0ec403e570 2024-05-10T20:56:23.197 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.195+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 msgr2=0x7f0ec4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.195+0000 7f0edfe00700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 0x7f0ec4040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006fd0 tx=0x7f0ecc006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.195+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 msgr2=0x7f0ee0075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.195+0000 7f0edfe00700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00092e0 tx=0x7f0ed0005780 comp rx=0 tx=0).stop 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 shutdown_connections 2024-05-10T20:56:23.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0ec403e570 0x7f0ec4040a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 --2- 172.21.15.50:0/1357486708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee01099a0 0x7f0ee0075ac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 >> 172.21.15.50:0/1357486708 conn(0x7f0ee00ff4b0 msgr2=0x7f0ee0100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 shutdown_connections 2024-05-10T20:56:23.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:23.196+0000 7f0edfe00700 1 -- 172.21.15.50:0/1357486708 wait complete. 2024-05-10T20:56:23.950 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T20:56:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:24.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:24.580 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:24 smithi050 ceph-mon[29546]: Added host smithi092 2024-05-10T20:56:24.580 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:24.580 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:24 smithi050 ceph-mon[29546]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:24.580 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:25.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:26.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:26 smithi050 ceph-mon[29546]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:26.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:26 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 -- 172.21.15.50:0/1408730307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 msgr2=0x7f9830102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 --2- 172.21.15.50:0/1408730307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f9830102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9820009910 tx=0x7f9820009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 -- 172.21.15.50:0/1408730307 shutdown_connections 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 --2- 172.21.15.50:0/1408730307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f9830102c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 -- 172.21.15.50:0/1408730307 >> 172.21.15.50:0/1408730307 conn(0x7f98300fe5b0 msgr2=0x7f9830100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 -- 172.21.15.50:0/1408730307 shutdown_connections 2024-05-10T20:56:26.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.756+0000 7f9837a00700 1 -- 172.21.15.50:0/1408730307 wait complete. 2024-05-10T20:56:26.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.758+0000 7f9837a00700 1 Processor -- start 2024-05-10T20:56:26.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.758+0000 7f9837a00700 1 -- start start 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9837a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9837a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9830110300 con 0x7f9830102880 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9835000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9835000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51018/0 (socket says 172.21.15.50:51018) 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9835000700 1 -- 172.21.15.50:0/1993226809 learned_addr learned my addr 172.21.15.50:0/1993226809 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:26.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.759+0000 7f9835000700 1 -- 172.21.15.50:0/1993226809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98301a1c90 con 0x7f9830102880 2024-05-10T20:56:26.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f9835000700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9820006010 tx=0x7f9820005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:26.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9820026070 con 0x7f9830102880 2024-05-10T20:56:26.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982000bc00 con 0x7f9830102880 2024-05-10T20:56:26.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9820003910 con 0x7f9830102880 2024-05-10T20:56:26.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98301a1f20 con 0x7f9830102880 2024-05-10T20:56:26.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.760+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983019b970 con 0x7f9830102880 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.761+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983006c100 con 0x7f9830102880 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.762+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9820029070 con 0x7f9830102880 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.762+0000 7f982e000700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 0x7f9814040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.762+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f982001e070 con 0x7f9830102880 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.762+0000 7f9827e00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 0x7f9814040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:26.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.763+0000 7f9827e00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 0x7f9814040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9818006fd0 tx=0x7f9818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:26.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:26.768+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9820013070 con 0x7f9830102880 2024-05-10T20:56:27.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.011+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f983019cea0 con 0x7f981403e510 2024-05-10T20:56:27.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.014+0000 7f982e000700 1 -- 172.21.15.50:0/1993226809 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f983019cea0 con 0x7f981403e510 2024-05-10T20:56:27.019 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:27.019 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-05-10T20:56:27.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 msgr2=0x7f9814040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:27.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 0x7f9814040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9818006fd0 tx=0x7f9818006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:27.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 msgr2=0x7f98301a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:27.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9820006010 tx=0x7f9820005780 comp rx=0 tx=0).stop 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 shutdown_connections 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f981403e510 0x7f9814040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 --2- 172.21.15.50:0/1993226809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9830102880 0x7f98301a1750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 >> 172.21.15.50:0/1993226809 conn(0x7f98300fe5b0 msgr2=0x7f983010c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 shutdown_connections 2024-05-10T20:56:27.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:27.017+0000 7f9837a00700 1 -- 172.21.15.50:0/1993226809 wait complete. 2024-05-10T20:56:27.784 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-05-10T20:56:27.784 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:56:27.784 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-05-10T20:56:27.803 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:56:27.803 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:27.860 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-05-10T20:56:27.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch host add smithi142 2024-05-10T20:56:28.327 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:28.737 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:28 smithi050 ceph-mon[29546]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:56:28.737 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:28 smithi050 ceph-mon[29546]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:30.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:30.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:30.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:30 smithi050 ceph-mon[29546]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.618+0000 7f40fa600700 1 -- 172.21.15.50:0/366786985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 msgr2=0x7f40f4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.618+0000 7f40fa600700 1 --2- 172.21.15.50:0/366786985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f4102d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009910 tx=0x7f40e4009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.619+0000 7f40fa600700 1 -- 172.21.15.50:0/366786985 shutdown_connections 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.619+0000 7f40fa600700 1 --2- 172.21.15.50:0/366786985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f4102d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.619+0000 7f40fa600700 1 -- 172.21.15.50:0/366786985 >> 172.21.15.50:0/366786985 conn(0x7f40f40785b0 msgr2=0x7f40f4078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.619+0000 7f40fa600700 1 -- 172.21.15.50:0/366786985 shutdown_connections 2024-05-10T20:56:30.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.619+0000 7f40fa600700 1 -- 172.21.15.50:0/366786985 wait complete. 2024-05-10T20:56:30.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.620+0000 7f40fa600700 1 Processor -- start 2024-05-10T20:56:30.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.620+0000 7f40fa600700 1 -- start start 2024-05-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.620+0000 7f40fa600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.620+0000 7f40fa600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4110350 con 0x7f40f4102910 2024-05-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.621+0000 7f40f3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.621+0000 7f40f3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51030/0 (socket says 172.21.15.50:51030) 2024-05-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.621+0000 7f40f3400700 1 -- 172.21.15.50:0/953993610 learned_addr learned my addr 172.21.15.50:0/953993610 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.621+0000 7f40f3400700 1 -- 172.21.15.50:0/953993610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f41a1ce0 con 0x7f40f4102910 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.621+0000 7f40f3400700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e40056d0 tx=0x7f40e40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.622+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40e4027070 con 0x7f40f4102910 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.622+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e400be10 con 0x7f40f4102910 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.622+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40e4022d20 con 0x7f40f4102910 2024-05-10T20:56:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.622+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41a1f70 con 0x7f40f4102910 2024-05-10T20:56:30.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.622+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f419b9c0 con 0x7f40f4102910 2024-05-10T20:56:30.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.623+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f406b670 con 0x7f40f4102910 2024-05-10T20:56:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.624+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f40e402a070 con 0x7f40f4102910 2024-05-10T20:56:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.624+0000 7f40ebe00700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 0x7f40d8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.624+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f40e401e030 con 0x7f40f4102910 2024-05-10T20:56:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.624+0000 7f40f2a00700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 0x7f40d8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.625+0000 7f40f2a00700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 0x7f40d8040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.630+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f40e4013070 con 0x7f40f4102910 2024-05-10T20:56:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:30.869+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) -- 0x7f40f419cef0 con 0x7f40d803e570 2024-05-10T20:56:31.545 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:31 smithi050 ceph-mon[29546]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:56:31.545 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:31.545 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:31.546 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:32.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:32 smithi050 ceph-mon[29546]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:32.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:32 smithi050 ceph-mon[29546]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:32.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:32 smithi050 ceph-mon[29546]: Deploying cephadm binary to smithi142 2024-05-10T20:56:34.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:34 smithi050 ceph-mon[29546]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:35.700 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-05-10T20:56:35.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.693+0000 7f40ebe00700 1 -- 172.21.15.50:0/953993610 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f40f419cef0 con 0x7f40d803e570 2024-05-10T20:56:35.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 msgr2=0x7f40d8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:35.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 0x7f40d8040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 msgr2=0x7f40f41a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e40056d0 tx=0x7f40e40057b0 comp rx=0 tx=0).stop 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 shutdown_connections 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f40d803e570 0x7f40d8040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 --2- 172.21.15.50:0/953993610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f4102910 0x7f40f41a17a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 >> 172.21.15.50:0/953993610 conn(0x7f40f40785b0 msgr2=0x7f40f41008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.698+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 shutdown_connections 2024-05-10T20:56:35.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:35.699+0000 7f40fa600700 1 -- 172.21.15.50:0/953993610 wait complete. 2024-05-10T20:56:36.391 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T20:56:36.849 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:37.002 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:36 smithi050 ceph-mon[29546]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:37.003 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:37.003 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:36 smithi050 ceph-mon[29546]: Added host smithi142 2024-05-10T20:56:37.003 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:37.003 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:38.122 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:37 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:38.122 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:37 smithi050 ceph-mon[29546]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:38.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.982+0000 7fe3c2600700 1 -- 172.21.15.50:0/856660518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 msgr2=0x7fe3bc108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:38.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.982+0000 7fe3c2600700 1 --2- 172.21.15.50:0/856660518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc108fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009910 tx=0x7fe3ac009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:38.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.982+0000 7fe3c2600700 1 -- 172.21.15.50:0/856660518 shutdown_connections 2024-05-10T20:56:38.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.982+0000 7fe3c2600700 1 --2- 172.21.15.50:0/856660518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc108fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:38.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.982+0000 7fe3c2600700 1 -- 172.21.15.50:0/856660518 >> 172.21.15.50:0/856660518 conn(0x7fe3bc0fde30 msgr2=0x7fe3bc100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:38.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.983+0000 7fe3c2600700 1 -- 172.21.15.50:0/856660518 shutdown_connections 2024-05-10T20:56:38.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.983+0000 7fe3c2600700 1 -- 172.21.15.50:0/856660518 wait complete. 2024-05-10T20:56:38.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.984+0000 7fe3c2600700 1 Processor -- start 2024-05-10T20:56:38.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.984+0000 7fe3c2600700 1 -- start start 2024-05-10T20:56:38.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.984+0000 7fe3c2600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:38.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.984+0000 7fe3c2600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10e210 con 0x7fe3bc108b90 2024-05-10T20:56:38.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.985+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:38.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.985+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57564/0 (socket says 172.21.15.50:57564) 2024-05-10T20:56:38.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.985+0000 7fe3bb400700 1 -- 172.21.15.50:0/4208098880 learned_addr learned my addr 172.21.15.50:0/4208098880 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.985+0000 7fe3bb400700 1 -- 172.21.15.50:0/4208098880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc104fb0 con 0x7fe3bc108b90 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.985+0000 7fe3bb400700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00b5c0 tx=0x7fe3ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.986+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac026070 con 0x7fe3bc108b90 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.986+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac00b750 con 0x7fe3bc108b90 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.986+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc1036a0 con 0x7fe3bc108b90 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.986+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac0037d0 con 0x7fe3bc108b90 2024-05-10T20:56:38.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.986+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc103aa0 con 0x7fe3bc108b90 2024-05-10T20:56:38.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.987+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc064e20 con 0x7fe3bc108b90 2024-05-10T20:56:38.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.987+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe3ac029070 con 0x7fe3bc108b90 2024-05-10T20:56:38.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.988+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 0x7fe3a0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:38.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.988+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe3ac01e070 con 0x7fe3bc108b90 2024-05-10T20:56:38.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.988+0000 7fe3baa00700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 0x7fe3a0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:38.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.989+0000 7fe3baa00700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 0x7fe3a0040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:38.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:38.994+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3ac013070 con 0x7fe3bc108b90 2024-05-10T20:56:39.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.236+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe3bc1052f0 con 0x7fe3a003e510 2024-05-10T20:56:39.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.239+0000 7fe3b3e00700 1 -- 172.21.15.50:0/4208098880 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fe3a800a040 con 0x7fe3a003e510 2024-05-10T20:56:39.244 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T20:56:39.244 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-05-10T20:56:39.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.242+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 msgr2=0x7fe3a0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:39.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.242+0000 7fe3c2600700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 0x7fe3a0040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:39.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.242+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 msgr2=0x7fe3bc103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:39.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.242+0000 7fe3c2600700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00b5c0 tx=0x7fe3ac005780 comp rx=0 tx=0).stop 2024-05-10T20:56:39.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 shutdown_connections 2024-05-10T20:56:39.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3a003e510 0x7fe3a0040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:39.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 --2- 172.21.15.50:0/4208098880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3bc108b90 0x7fe3bc103160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:39.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 >> 172.21.15.50:0/4208098880 conn(0x7fe3bc0fde30 msgr2=0x7fe3bc105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:39.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 shutdown_connections 2024-05-10T20:56:39.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:39.243+0000 7fe3c2600700 1 -- 172.21.15.50:0/4208098880 wait complete. 2024-05-10T20:56:39.624 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:40.011 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T20:56:40.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T20:56:40.454 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:56:40.732 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:40 smithi050 ceph-mon[29546]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:56:40.732 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:40 smithi050 ceph-mon[29546]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 -- 172.21.15.50:0/1673784703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 msgr2=0x7f9744104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 --2- 172.21.15.50:0/1673784703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f9744104fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9734009910 tx=0x7f9734009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 -- 172.21.15.50:0/1673784703 shutdown_connections 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 --2- 172.21.15.50:0/1673784703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f9744104fa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 -- 172.21.15.50:0/1673784703 >> 172.21.15.50:0/1673784703 conn(0x7f97440fe3e0 msgr2=0x7f9744100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:42.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 -- 172.21.15.50:0/1673784703 shutdown_connections 2024-05-10T20:56:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.545+0000 7f9743e00700 1 -- 172.21.15.50:0/1673784703 wait complete. 2024-05-10T20:56:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.546+0000 7f9743e00700 1 Processor -- start 2024-05-10T20:56:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.546+0000 7f9743e00700 1 -- start start 2024-05-10T20:56:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.547+0000 7f9743e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.547+0000 7f9743e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f974410df80 con 0x7f9744104b90 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.547+0000 7f9742a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.547+0000 7f9742a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57592/0 (socket says 172.21.15.50:57592) 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.547+0000 7f9742a00700 1 -- 172.21.15.50:0/1038897731 learned_addr learned my addr 172.21.15.50:0/1038897731 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f9742a00700 1 -- 172.21.15.50:0/1038897731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419bfd0 con 0x7f9744104b90 2024-05-10T20:56:42.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f9742a00700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97340056d0 tx=0x7f97340057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9734026070 con 0x7f9744104b90 2024-05-10T20:56:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419c260 con 0x7f9744104b90 2024-05-10T20:56:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9734004370 con 0x7f9744104b90 2024-05-10T20:56:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.548+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97340039b0 con 0x7f9744104b90 2024-05-10T20:56:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.549+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974419c6c0 con 0x7f9744104b90 2024-05-10T20:56:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.550+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974406c1e0 con 0x7f9744104b90 2024-05-10T20:56:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.550+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9734029070 con 0x7f9744104b90 2024-05-10T20:56:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.551+0000 7f973b400700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 0x7f9728040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.551+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f973401e070 con 0x7f9744104b90 2024-05-10T20:56:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.551+0000 7f9742000700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 0x7f9728040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:42.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.552+0000 7f9742000700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 0x7f9728040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:42.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.557+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9734013070 con 0x7f9744104b90 2024-05-10T20:56:42.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:42.800+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f974419cdd0 con 0x7f9744104b90 2024-05-10T20:56:42.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:42.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:42.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:42.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:42.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:42.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:56:42.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:42.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:42.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:42 smithi050 ceph-mon[29546]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:56:43.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.684+0000 7f973b400700 1 -- 172.21.15.50:0/1038897731 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f973400bcc0 con 0x7f9744104b90 2024-05-10T20:56:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.688+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 msgr2=0x7f9728040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:43.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 0x7f9728040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 msgr2=0x7f974419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97340056d0 tx=0x7f97340057b0 comp rx=0 tx=0).stop 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 shutdown_connections 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f972803e510 0x7f9728040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 --2- 172.21.15.50:0/1038897731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9744104b90 0x7f974419ba90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 >> 172.21.15.50:0/1038897731 conn(0x7f97440fe3e0 msgr2=0x7f9744109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 shutdown_connections 2024-05-10T20:56:43.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:56:43.689+0000 7f9743e00700 1 -- 172.21.15.50:0/1038897731 wait complete. 2024-05-10T20:56:43.696 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2024-05-10T20:56:44.070 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:44.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:44.461 INFO:tasks.cephadm:Adding mon.a on smithi050 2024-05-10T20:56:44.461 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-05-10T20:56:44.461 INFO:tasks.cephadm:Adding mon.c on smithi142 2024-05-10T20:56:44.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c' 2024-05-10T20:56:44.698 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:45.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:44 smithi050 ceph-mon[29546]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:45.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:44 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T20:56:45.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:44 smithi050 ceph-mon[29546]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:56:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 -- 172.21.15.142:0/4243079228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 msgr2=0x7f4ab0103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 --2- 172.21.15.142:0/4243079228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab0103830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009910 tx=0x7f4aa0009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 -- 172.21.15.142:0/4243079228 shutdown_connections 2024-05-10T20:56:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 --2- 172.21.15.142:0/4243079228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab0103830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 -- 172.21.15.142:0/4243079228 >> 172.21.15.142:0/4243079228 conn(0x7f4ab0077d30 msgr2=0x7f4ab0078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 -- 172.21.15.142:0/4243079228 shutdown_connections 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.020+0000 7f4ab616e700 1 -- 172.21.15.142:0/4243079228 wait complete. 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.021+0000 7f4ab616e700 1 Processor -- start 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.021+0000 7f4ab616e700 1 -- start start 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.021+0000 7f4ab616e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.021+0000 7f4ab616e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab010d1f0 con 0x7f4ab0103420 2024-05-10T20:56:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.022+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.022+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:57180/0 (socket says 172.21.15.142:57180) 2024-05-10T20:56:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.022+0000 7f4aaf7fe700 1 -- 172.21.15.142:0/3162300992 learned_addr learned my addr 172.21.15.142:0/3162300992 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.022+0000 7f4aaf7fe700 1 -- 172.21.15.142:0/3162300992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab019fcc0 con 0x7f4ab0103420 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4aaf7fe700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00056d0 tx=0x7f4aa00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa0026070 con 0x7f4ab0103420 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019ffb0 con 0x7f4ab0103420 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01999a0 con 0x7f4ab0103420 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa000b6f0 con 0x7f4ab0103420 2024-05-10T20:56:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.023+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa0030060 con 0x7f4ab0103420 2024-05-10T20:56:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.024+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c005330 con 0x7f4ab0103420 2024-05-10T20:56:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.025+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4aa0029070 con 0x7f4ab0103420 2024-05-10T20:56:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.025+0000 7f4aacff9700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 0x7f4a98040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.025+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4aa001e070 con 0x7f4ab0103420 2024-05-10T20:56:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.025+0000 7f4aaeffd700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 0x7f4a98040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:46.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.026+0000 7f4aaeffd700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 0x7f4a98040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4006fd0 tx=0x7f4aa4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:46.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.041+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4aa0013070 con 0x7f4ab0103420 2024-05-10T20:56:46.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.255+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}) -- 0x7f4a9c000c80 con 0x7f4a9803e510 2024-05-10T20:56:46.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.271+0000 7f4aacff9700 1 -- 172.21.15.142:0/3162300992 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4a9c000c80 con 0x7f4a9803e510 2024-05-10T20:56:46.277 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 msgr2=0x7f4a98040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 0x7f4a98040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4006fd0 tx=0x7f4aa4006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 msgr2=0x7f4ab019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00056d0 tx=0x7f4aa00057b0 comp rx=0 tx=0).stop 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 shutdown_connections 2024-05-10T20:56:46.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a9803e510 0x7f4a98040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:46.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 --2- 172.21.15.142:0/3162300992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ab0103420 0x7f4ab019f780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:46.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 >> 172.21.15.142:0/3162300992 conn(0x7f4ab0077d30 msgr2=0x7f4ab0102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:46.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 shutdown_connections 2024-05-10T20:56:46.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:46.274+0000 7f4ab616e700 1 -- 172.21.15.142:0/3162300992 wait complete. 2024-05-10T20:56:46.833 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T20:56:46.836 DEBUG:teuthology.orchestra.run.smithi142:mon.c> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T20:56:46.838 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T20:56:46.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T20:56:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:47.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:47.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:56:47.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:47.513 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:48.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:47 smithi050 ceph-mon[29546]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:48.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:47 smithi050 ceph-mon[29546]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c;count:3 2024-05-10T20:56:48.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:47 smithi050 ceph-mon[29546]: Deploying daemon mon.c on smithi142 2024-05-10T20:56:49.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:48 smithi050 ceph-mon[29546]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:50.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 -- 172.21.15.142:0/353451802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 msgr2=0x7f22f4108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:50.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 --2- 172.21.15.142:0/353451802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f4108ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009910 tx=0x7f22e4009c20 comp rx=0 tx=0).stop 2024-05-10T20:56:50.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 -- 172.21.15.142:0/353451802 shutdown_connections 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 --2- 172.21.15.142:0/353451802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f4108ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 -- 172.21.15.142:0/353451802 >> 172.21.15.142:0/353451802 conn(0x7f22f40785b0 msgr2=0x7f22f4078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 -- 172.21.15.142:0/353451802 shutdown_connections 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.570+0000 7f22fc740700 1 -- 172.21.15.142:0/353451802 wait complete. 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.571+0000 7f22fc740700 1 Processor -- start 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.571+0000 7f22fc740700 1 -- start start 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.571+0000 7f22fc740700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.571+0000 7f22fc740700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410cfe0 con 0x7f22f4108a90 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.572+0000 7f22fa4dc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:50.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.572+0000 7f22fa4dc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:57192/0 (socket says 172.21.15.142:57192) 2024-05-10T20:56:50.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.572+0000 7f22fa4dc700 1 -- 172.21.15.142:0/374890232 learned_addr learned my addr 172.21.15.142:0/374890232 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:56:50.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.572+0000 7f22fa4dc700 1 -- 172.21.15.142:0/374890232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f419ea30 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22fa4dc700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f22e40056d0 tx=0x7f22e40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e4027070 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f419ed20 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f4198850 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e400be10 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.573+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e4031060 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.574+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f41041a0 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.575+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f22e402a070 con 0x7f22f4108a90 2024-05-10T20:56:50.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.575+0000 7f22eb7fe700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 0x7f22e0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:50.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.575+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f22e401e030 con 0x7f22f4108a90 2024-05-10T20:56:50.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.576+0000 7f22f9cdb700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 0x7f22e0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:50.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.577+0000 7f22f9cdb700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 0x7f22e0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:50.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.592+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22e4013070 con 0x7f22f4108a90 2024-05-10T20:56:50.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.911+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f22f419ef50 con 0x7f22f4108a90 2024-05-10T20:56:50.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.912+0000 7f22eb7fe700 1 -- 172.21.15.142:0/374890232 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f22e401f2d0 con 0x7f22f4108a90 2024-05-10T20:56:50.917 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:56:50.917 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","modified":"2024-05-10T20:54:31.773579Z","created":"2024-05-10T20:54:31.773579Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T20:56:50.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 msgr2=0x7f22e0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:50.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 0x7f22e0040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0006e40 comp rx=0 tx=0).stop 2024-05-10T20:56:50.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 msgr2=0x7f22f419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:50.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f22e40056d0 tx=0x7f22e40057b0 comp rx=0 tx=0).stop 2024-05-10T20:56:50.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 shutdown_connections 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f22e003e510 0x7f22e0040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 --2- 172.21.15.142:0/374890232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22f4108a90 0x7f22f419e4f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 >> 172.21.15.142:0/374890232 conn(0x7f22f40785b0 msgr2=0x7f22f40ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 shutdown_connections 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:50.914+0000 7f22fc740700 1 -- 172.21.15.142:0/374890232 wait complete. 2024-05-10T20:56:50.920 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-05-10T20:56:51.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 systemd[1]: Starting Ceph mon.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T20:56:51.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:50 smithi050 ceph-mon[29546]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 podman[34827]: 2024-05-10 20:56:51.752279969 +0000 UTC m=+0.086343812 container create cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 podman[34827]: 2024-05-10 20:56:51.683328598 +0000 UTC m=+0.017392420 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 podman[34827]: 2024-05-10 20:56:51.955105445 +0000 UTC m=+0.289169257 container init cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True) 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 podman[34827]: 2024-05-10 20:56:51.958181206 +0000 UTC m=+0.292245005 container start cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev), process ceph-mon, pid 2 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: pidfile_write: ignore empty --pid-file 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: load: jerasure load: lrc 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: RocksDB version: 7.9.2 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Git sha 0 2024-05-10T20:56:52.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Compile date 2024-05-09 13:50:00 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: DB SUMMARY 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: DB Session ID: OM2LNDZ9M5F8IWYR2XP2 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: CURRENT file: CURRENT 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T20:56:52.027 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.error_if_exists: 0 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.create_if_missing: 0 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.paranoid_checks: 1 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.env: 0x55d903312500 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T20:56:52.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.info_log: 0x55d903d02020 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.statistics: (nil) 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.use_fsync: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_log_file_size: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_fallocate: 1 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.use_direct_reads: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T20:56:52.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.db_log_dir: 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.wal_dir: 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.write_buffer_manager: 0x55d903bcc8c0 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T20:56:52.030 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.unordered_write: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.row_cache: None 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.wal_filter: None 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T20:56:52.031 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.two_write_queues: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.wal_compression: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.atomic_flush: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.log_readahead_size: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T20:56:52.032 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_background_jobs: 2 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_background_compactions: -1 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_subcompactions: 1 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_open_files: -1 2024-05-10T20:56:52.033 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_background_flushes: -1 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Compression algorithms supported: 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kZSTD supported: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kXpressCompression supported: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kZlibCompression supported: 1 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kSnappyCompression supported: 1 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kLZ4Compression supported: 1 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: kBZip2Compression supported: 0 2024-05-10T20:56:52.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.merge_operator: 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_filter: None 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d903d021e0) 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-05-10T20:56:52.035 INFO:journalctl@ceph.mon.c.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_type: 0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: data_block_index_type: 0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_shortening: 1 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: checksum: 4 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: no_block_cache: 0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache: 0x55d903bff1f0 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_options: 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: capacity : 536870912 2024-05-10T20:56:52.036 INFO:journalctl@ceph.mon.c.smithi142.stdout: num_shard_bits : 4 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: strict_capacity_limit : 0 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_compressed: (nil) 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: persistent_cache: (nil) 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_size: 4096 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_size_deviation: 10 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_restart_interval: 16 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_block_restart_interval: 1 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: metadata_block_size: 4096 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: partition_filters: 0 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: use_delta_encoding: 1 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: filter_policy: bloomfilter 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: whole_key_filtering: 1 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: verify_compression: 0 2024-05-10T20:56:52.037 INFO:journalctl@ceph.mon.c.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: format_version: 5 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: enable_index_compression: 1 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_align: 0 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: max_auto_readahead_size: 262144 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: prepopulate_block_cache: 0 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: initial_auto_readahead_size: 8192 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression: NoCompression 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T20:56:52.038 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.num_levels: 7 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T20:56:52.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T20:56:52.040 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T20:56:52.040 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T20:56:52.040 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:56:52.040 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:56:52.040 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T20:56:52.041 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T20:56:52.042 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.inplace_update_support: 0 2024-05-10T20:56:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.bloom_locality: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.max_successive_merges: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.ttl: 2592000 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T20:56:52.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enable_blob_files: false 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.min_blob_size: 0 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6d8887cc-351e-45ca-94d8-989c95965348 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374611998842, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T20:56:52.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374611999293, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715374611, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6d8887cc-351e-45ca-94d8-989c95965348", "db_session_id": "OM2LNDZ9M5F8IWYR2XP2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374611999374, "job": 1, "event": "recovery_finished"} 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:51 smithi142 ceph-mon[34846]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d903be3c00 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: rocksdb: DB pointer 0x55d9089d6000 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** DB Stats ** 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:56:52.046 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T20:56:52.047 INFO:journalctl@ceph.mon.c.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache BinnedLRUCache@0x55d903bff1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-10T20:56:52.048 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: starting mon.c rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(???) e0 preinit fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 bash[34827]: cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 2024-05-10T20:56:52.049 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 systemd[1]: Started Ceph mon.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:56:52.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:51 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/374890232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:56:52.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: e1 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: legacy client fscid: -1 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout: No filesystems configured 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T20:56:52.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Deploying cephadm binary to smithi092 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Added host smithi092 2024-05-10T20:56:52.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:52.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Deploying cephadm binary to smithi142 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Added host smithi142 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:52.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:52.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c;count:3 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: Deploying daemon mon.c on smithi142 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/374890232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:56:52.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:52 smithi142 ceph-mon[34846]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T20:56:52.533 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T20:56:52.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T20:56:52.769 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:56:57.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.247+0000 7fe8bebe6700 1 -- 172.21.15.142:0/996566819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 msgr2=0x7fe8a4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.247+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/996566819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8a4005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009bf0 tx=0x7fe8a8009f00 comp rx=0 tx=0).stop 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 -- 172.21.15.142:0/996566819 shutdown_connections 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/996566819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8a4005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 -- 172.21.15.142:0/996566819 >> 172.21.15.142:0/996566819 conn(0x7fe8b80fdfa0 msgr2=0x7fe8b81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 -- 172.21.15.142:0/996566819 shutdown_connections 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 -- 172.21.15.142:0/996566819 wait complete. 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 Processor -- start 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.248+0000 7fe8bebe6700 1 -- start start 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8bebe6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8b8104d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8bebe6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8bebe6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81099c0 con 0x7fe8b8102510 2024-05-10T20:56:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8bebe6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8109840 con 0x7fe8b81052d0 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56288/0 (socket says 172.21.15.142:56288) 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 -- 172.21.15.142:0/1773268370 learned_addr learned my addr 172.21.15.142:0/1773268370 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 -- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 msgr2=0x7fe8b81056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 -- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 msgr2=0x7fe8b81056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T20:56:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8b7fff700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.249+0000 7fe8bc982700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8b8104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.250+0000 7fe8bc982700 1 -- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 msgr2=0x7fe8b81056e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.250+0000 7fe8bc982700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.250+0000 7fe8bc982700 1 -- 172.21.15.142:0/1773268370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b8106cc0 con 0x7fe8b8102510 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.250+0000 7fe8bc982700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8b8104d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000f80 tx=0x7fe8a80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.251+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe8a8017470 con 0x7fe8b8102510 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.251+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b81a5d50 con 0x7fe8b8102510 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.251+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a6200 con 0x7fe8b8102510 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.251+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a8024030 con 0x7fe8b8102510 2024-05-10T20:56:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.251+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe8a80206a0 con 0x7fe8b8102510 2024-05-10T20:56:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.252+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b81039c0 con 0x7fe8b8102510 2024-05-10T20:56:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.253+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe8a801e070 con 0x7fe8b8102510 2024-05-10T20:56:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.253+0000 7fe8b5ffb700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 0x7fe8980407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:56:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.253+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8a801c070 con 0x7fe8b8102510 2024-05-10T20:56:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.254+0000 7fe8b7fff700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 0x7fe8980407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:56:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.255+0000 7fe8b7fff700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 0x7fe8980407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8100ca0 tx=0x7fe8a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:56:57.272 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.266+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8a8025280 con 0x7fe8b8102510 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: mon.a calling monitor election 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: mon.c calling monitor election 2024-05-10T20:56:57.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: monmap epoch 2 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: last_changed 2024-05-10T20:56:52.036678+0000 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: min_mon_release 19 (squid) 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: election_strategy: 1 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: fsmap 2024-05-10T20:56:57.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: mgrmap e13: x(active, since 61s) 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: overall HEALTH_OK 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:56:57.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:57.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:56:57.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: mon.a calling monitor election 2024-05-10T20:56:57.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: mon.c calling monitor election 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: monmap epoch 2 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: last_changed 2024-05-10T20:56:52.036678+0000 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: min_mon_release 19 (squid) 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: election_strategy: 1 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:56:57.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: fsmap 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: mgrmap e13: x(active, since 61s) 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: overall HEALTH_OK 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:56:57.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:56:57.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.556+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe8b802e7d0 con 0x7fe8b8102510 2024-05-10T20:56:57.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.557+0000 7fe8b5ffb700 1 -- 172.21.15.142:0/1773268370 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe8a801e360 con 0x7fe8b8102510 2024-05-10T20:56:57.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:56:57.563 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","modified":"2024-05-10T20:56:52.036678Z","created":"2024-05-10T20:54:31.773579Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 msgr2=0x7fe8980407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 0x7fe8980407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8100ca0 tx=0x7fe8a0008040 comp rx=0 tx=0).stop 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 msgr2=0x7fe8b8104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8b8104d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000f80 tx=0x7fe8a80044b0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 shutdown_connections 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe89803e2b0 0x7fe8980407c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8b8102510 0x7fe8b8104d90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 --2- 172.21.15.142:0/1773268370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b81052d0 0x7fe8b81056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 >> 172.21.15.142:0/1773268370 conn(0x7fe8b80fdfa0 msgr2=0x7fe8b810b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 shutdown_connections 2024-05-10T20:56:57.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:56:57.559+0000 7fe8bebe6700 1 -- 172.21.15.142:0/1773268370 wait complete. 2024-05-10T20:56:57.567 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-05-10T20:56:58.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:58 smithi050 ceph-mon[29546]: Deploying daemon mon.b on smithi092 2024-05-10T20:56:58.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:58 smithi050 ceph-mon[29546]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:58.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:56:58 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/1773268370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:56:58.424 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:56:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:56:58.055+0000 7fc394200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-10T20:56:58.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:58 smithi142 ceph-mon[34846]: Deploying daemon mon.b on smithi092 2024-05-10T20:56:58.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:58 smithi142 ceph-mon[34846]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:56:58.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:56:58 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/1773268370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:56:59.172 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T20:56:59.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T20:56:59.415 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:57:00.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:00 smithi142 ceph-mon[34846]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:00.839 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:00 smithi092 podman[34394]: 2024-05-10 20:57:00.7373588 +0000 UTC m=+0.018234381 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T20:57:00.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 -- 172.21.15.142:0/930153150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 msgr2=0x7f3e281051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:00.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 --2- 172.21.15.142:0/930153150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e281051e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009910 tx=0x7f3e18009c20 comp rx=0 tx=0).stop 2024-05-10T20:57:00.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 -- 172.21.15.142:0/930153150 shutdown_connections 2024-05-10T20:57:00.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 --2- 172.21.15.142:0/930153150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e281051e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:00.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 -- 172.21.15.142:0/930153150 >> 172.21.15.142:0/930153150 conn(0x7f3e280797e0 msgr2=0x7f3e28079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:00.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.884+0000 7f3e2e045700 1 -- 172.21.15.142:0/930153150 shutdown_connections 2024-05-10T20:57:00.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.885+0000 7f3e2e045700 1 -- 172.21.15.142:0/930153150 wait complete. 2024-05-10T20:57:00.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.885+0000 7f3e2e045700 1 Processor -- start 2024-05-10T20:57:00.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e2e045700 1 -- start start 2024-05-10T20:57:00.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e2e045700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e2819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:00.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e2e045700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:00.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e2e045700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2810e230 con 0x7f3e28104dd0 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e2e045700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2810e0b0 con 0x7f3e2819fd90 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56304/0 (socket says 172.21.15.142:56304) 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 -- 172.21.15.142:0/1661305991 learned_addr learned my addr 172.21.15.142:0/1661305991 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 -- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 msgr2=0x7f3e28199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 -- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 msgr2=0x7f3e28199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e26ffd700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.886+0000 7f3e277fe700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e2819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.887+0000 7f3e277fe700 1 -- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 msgr2=0x7f3e28199a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T20:57:00.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.887+0000 7f3e277fe700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:00.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.887+0000 7f3e277fe700 1 -- 172.21.15.142:0/1661305991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e281a0360 con 0x7f3e28104dd0 2024-05-10T20:57:00.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e277fe700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e2819f850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e18014000 tx=0x7f3e18005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3e18005dc0 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e28199fb0 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2819a540 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1800be10 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.888+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3e1800fe60 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.889+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2806c560 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.890+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3e1802a070 con 0x7f3e28104dd0 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.890+0000 7f3e24ff9700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 0x7f3e100407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:00.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.890+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e1801e030 con 0x7f3e28104dd0 2024-05-10T20:57:00.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.891+0000 7f3e26ffd700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 0x7f3e100407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:00.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.892+0000 7f3e26ffd700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 0x7f3e100407c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e28107050 tx=0x7f3e1c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:00.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:00.906+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e18031080 con 0x7f3e28104dd0 2024-05-10T20:57:00.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:00 smithi050 ceph-mon[29546]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.107 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 podman[34394]: 2024-05-10 20:57:01.066419606 +0000 UTC m=+0.347295169 container init a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-10T20:57:01.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 podman[34394]: 2024-05-10 20:57:01.069530203 +0000 UTC m=+0.350405767 container start a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-10T20:57:01.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:01.137+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e180047d0 con 0x7f3e28104dd0 2024-05-10T20:57:01.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:01.193+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3e2819afa0 con 0x7f3e28104dd0 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: ceph version 19.0.0-3590-g7b710596 (7b710596db3f44de0a4b72eff459db857656fe05) squid (dev), process ceph-mon, pid 2 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pidfile_write: ignore empty --pid-file 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: load: jerasure load: lrc 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: RocksDB version: 7.9.2 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Git sha 0 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Compile date 2024-05-09 13:50:00 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: DB SUMMARY 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: DB Session ID: 3KSA7LKY4NCW2QD91D2V 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: CURRENT file: CURRENT 2024-05-10T20:57:01.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.error_if_exists: 0 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.create_if_missing: 0 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.paranoid_checks: 1 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.env: 0x5596eaec2500 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.info_log: 0x5596ebe86020 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.statistics: (nil) 2024-05-10T20:57:01.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.use_fsync: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_log_file_size: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_fallocate: 1 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.use_direct_reads: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.db_log_dir: 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.wal_dir: 2024-05-10T20:57:01.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.write_buffer_manager: 0x5596ebd508c0 2024-05-10T20:57:01.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.unordered_write: 0 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.row_cache: None 2024-05-10T20:57:01.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.wal_filter: None 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.two_write_queues: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.wal_compression: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.atomic_flush: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.log_readahead_size: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T20:57:01.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_background_jobs: 2 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_background_compactions: -1 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_subcompactions: 1 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T20:57:01.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_open_files: -1 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_background_flushes: -1 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Compression algorithms supported: 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kZSTD supported: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kXpressCompression supported: 0 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kZlibCompression supported: 1 2024-05-10T20:57:01.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kSnappyCompression supported: 1 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kLZ4Compression supported: 1 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: kBZip2Compression supported: 0 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.merge_operator: 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_filter: None 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5596ebe861e0) 2024-05-10T20:57:01.384 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_type: 0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_index_type: 0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_shortening: 1 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: checksum: 4 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: no_block_cache: 0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache: 0x5596ebd831f0 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-05-10T20:57:01.385 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_options: 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: capacity : 536870912 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_shard_bits : 4 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: strict_capacity_limit : 0 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_cache_compressed: (nil) 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: persistent_cache: (nil) 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size: 4096 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_size_deviation: 10 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_restart_interval: 16 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: index_block_restart_interval: 1 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: metadata_block_size: 4096 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: partition_filters: 0 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: use_delta_encoding: 1 2024-05-10T20:57:01.386 INFO:journalctl@ceph.mon.b.smithi092.stdout: filter_policy: bloomfilter 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: whole_key_filtering: 1 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: verify_compression: 0 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: format_version: 5 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_index_compression: 1 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: block_align: 0 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: max_auto_readahead_size: 262144 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: prepopulate_block_cache: 0 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: initial_auto_readahead_size: 8192 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T20:57:01.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression: NoCompression 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.num_levels: 7 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T20:57:01.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T20:57:01.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T20:57:01.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T20:57:01.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.inplace_update_support: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.bloom_locality: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.max_successive_merges: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T20:57:01.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.ttl: 2592000 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enable_blob_files: false 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.min_blob_size: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T20:57:01.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e3160390-d0ac-41b0-ae45-926ec3037dd2 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374621109840, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374621110270, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715374621, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e3160390-d0ac-41b0-ae45-926ec3037dd2", "db_session_id": "3KSA7LKY4NCW2QD91D2V", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715374621110350, "job": 1, "event": "recovery_finished"} 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5596ebd67c00 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: DB pointer 0x5596f0b5a000 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T20:57:01.394 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** DB Stats ** 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:57:01.395 INFO:journalctl@ceph.mon.b.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T20:57:01.396 INFO:journalctl@ceph.mon.b.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache BinnedLRUCache@0x5596ebd831f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: using public_addr v2:172.21.15.92:0/0 -> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: starting mon.b rank -1 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:01.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(???) e0 preinit fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:01.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).mds e1 new map 2024-05-10T20:57:01.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-10T20:57:01.398 INFO:journalctl@ceph.mon.b.smithi092.stdout: e1 2024-05-10T20:57:01.398 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T20:57:01.398 INFO:journalctl@ceph.mon.b.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout: legacy client fscid: -1 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout: No filesystems configured 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:57:01.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T20:57:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Deploying cephadm binary to smithi092 2024-05-10T20:57:01.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Added host smithi092 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.401 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:57:01.402 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Deploying cephadm binary to smithi142 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Added host smithi142 2024-05-10T20:57:01.403 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:01.404 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1038897731' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.405 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c;count:3 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Deploying daemon mon.c on smithi142 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/374890232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:57:01.406 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.a calling monitor election 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.c calling monitor election 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: monmap epoch 2 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: last_changed 2024-05-10T20:56:52.036678+0000 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: min_mon_release 19 (squid) 2024-05-10T20:57:01.407 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: election_strategy: 1 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: fsmap 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mgrmap e13: x(active, since 61s) 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: overall HEALTH_OK 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: Deploying daemon mon.b on smithi092 2024-05-10T20:57:01.408 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/1773268370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:57:01.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:01.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 ceph-mon[34415]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T20:57:01.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 bash[34394]: a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 2024-05-10T20:57:01.410 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:01 smithi092 systemd[1]: Started Ceph mon.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:57:06.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-10T20:57:06.171 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":3,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","modified":"2024-05-10T20:57:01.135838Z","created":"2024-05-10T20:54:31.773579Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-10T20:57:06.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.165+0000 7f3e24ff9700 1 -- 172.21.15.142:0/1661305991 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f3e18013070 con 0x7f3e28104dd0 2024-05-10T20:57:06.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 msgr2=0x7f3e100407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:06.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 0x7f3e100407c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e28107050 tx=0x7f3e1c008040 comp rx=0 tx=0).stop 2024-05-10T20:57:06.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 msgr2=0x7f3e2819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:06.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e2819f850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e18014000 tx=0x7f3e18005030 comp rx=0 tx=0).stop 2024-05-10T20:57:06.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 shutdown_connections 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e1003e2b0 0x7f3e100407c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e28104dd0 0x7f3e2819f850 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 --2- 172.21.15.142:0/1661305991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e2819fd90 0x7f3e28199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 >> 172.21.15.142:0/1661305991 conn(0x7f3e280797e0 msgr2=0x7f3e2810a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 shutdown_connections 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:06.168+0000 7f3e2e045700 1 -- 172.21.15.142:0/1661305991 wait complete. 2024-05-10T20:57:06.174 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 3 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: mon.a calling monitor election 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: mon.c calling monitor election 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/1661305991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: mon.b calling monitor election 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.482 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: monmap epoch 3 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: last_changed 2024-05-10T20:57:01.135838+0000 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: min_mon_release 19 (squid) 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: election_strategy: 1 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-10T20:57:06.483 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: fsmap 2024-05-10T20:57:06.484 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:57:06.484 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: mgrmap e13: x(active, since 70s) 2024-05-10T20:57:06.484 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: overall HEALTH_OK 2024-05-10T20:57:06.484 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.484 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:57:06.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: mon.a calling monitor election 2024-05-10T20:57:06.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: mon.c calling monitor election 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/1661305991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: mon.b calling monitor election 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: monmap epoch 3 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: last_changed 2024-05-10T20:57:01.135838+0000 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: min_mon_release 19 (squid) 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: election_strategy: 1 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: fsmap 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: mgrmap e13: x(active, since 70s) 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: overall HEALTH_OK 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:57:06.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: mon.a calling monitor election 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: mon.c calling monitor election 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/1661305991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: mon.b calling monitor election 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:06.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: monmap epoch 3 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: last_changed 2024-05-10T20:57:01.135838+0000 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: created 2024-05-10T20:54:31.773579+0000 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: min_mon_release 19 (squid) 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: election_strategy: 1 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: fsmap 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: mgrmap e13: x(active, since 70s) 2024-05-10T20:57:06.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: overall HEALTH_OK 2024-05-10T20:57:06.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:06.786 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T20:57:06.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T20:57:07.235 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:57:07.503 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:07 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.504 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:07 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.504 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:07 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:07.504 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:07 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:07.524 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:07 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.524 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:07 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.524 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:07 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:07.524 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:07 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:07.544 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:07 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.544 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:07 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:07.544 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:07 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:07.544 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:07 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:57:08.407 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 20:57:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T20:57:08.134+0000 7fc394200700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:08.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:08.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:08.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:08.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:09.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.420+0000 7fa2d0000700 1 -- 172.21.15.50:0/693092903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 msgr2=0x7fa2c8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:09.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.420+0000 7fa2d0000700 1 --2- 172.21.15.50:0/693092903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c8108e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009910 tx=0x7fa2bc009c20 comp rx=0 tx=0).stop 2024-05-10T20:57:09.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.420+0000 7fa2d0000700 1 -- 172.21.15.50:0/693092903 shutdown_connections 2024-05-10T20:57:09.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.420+0000 7fa2d0000700 1 --2- 172.21.15.50:0/693092903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c8108e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.420+0000 7fa2d0000700 1 -- 172.21.15.50:0/693092903 >> 172.21.15.50:0/693092903 conn(0x7fa2c80fe5b0 msgr2=0x7fa2c8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:09.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.421+0000 7fa2d0000700 1 -- 172.21.15.50:0/693092903 shutdown_connections 2024-05-10T20:57:09.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.421+0000 7fa2d0000700 1 -- 172.21.15.50:0/693092903 wait complete. 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.422+0000 7fa2d0000700 1 Processor -- start 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.422+0000 7fa2d0000700 1 -- start start 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.422+0000 7fa2d0000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.422+0000 7fa2d0000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c819c380 0x7fa2c81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2d0000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 0x7fa2c81a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2d0000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c810cfb0 con 0x7fa2c8108a60 2024-05-10T20:57:09.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2d0000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c810ce30 con 0x7fa2c819c380 2024-05-10T20:57:09.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2d0000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c810d130 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2cd600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:09.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2cd600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37836/0 (socket says 172.21.15.50:37836) 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2cd600700 1 -- 172.21.15.50:0/3134331257 learned_addr learned my addr 172.21.15.50:0/3134331257 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2ce000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 0x7fa2c81a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.423+0000 7fa2ccc00700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c819c380 0x7fa2c81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 -- 172.21.15.50:0/3134331257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c819c380 msgr2=0x7fa2c81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c819c380 0x7fa2c81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 -- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 msgr2=0x7fa2c819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 -- 172.21.15.50:0/3134331257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c81a8920 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2ce000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 0x7fa2c81a0fc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000bd50 tx=0x7fa2c000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0019070 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c000cbb0 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00125a0 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.424+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c81a8b50 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.425+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c81a9040 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.426+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c8064e20 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.427+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa2c001c070 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.428+0000 7fa2c6000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 0x7fa2b0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:09.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.428+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa2c0052000 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.428+0000 7fa2cd600700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 0x7fa2b0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:09.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.429+0000 7fa2cd600700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 0x7fa2b0040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009750 tx=0x7fa2bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:09.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.433+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2c0012740 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.693 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: Updating smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.694 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.687+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa2c802e7b0 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.695 INFO:teuthology.orchestra.run.smithi050.stdout:# minimal ceph.conf for 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:09.695 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-05-10T20:57:09.695 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T20:57:09.696 INFO:teuthology.orchestra.run.smithi050.stdout: mon_host = [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-05-10T20:57:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.688+0000 7fa2c6000700 1 -- 172.21.15.50:0/3134331257 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa2c000cd60 con 0x7fa2c81a0bb0 2024-05-10T20:57:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 msgr2=0x7fa2b0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 0x7fa2b0040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009750 tx=0x7fa2bc01a040 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 msgr2=0x7fa2c81a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 0x7fa2c81a0fc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000bd50 tx=0x7fa2c000bd80 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 shutdown_connections 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa2b003e9a0 0x7fa2b0040eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2c81a0bb0 0x7fa2c81a0fc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c8108a60 0x7fa2c819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 --2- 172.21.15.50:0/3134331257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c819c380 0x7fa2c81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 >> 172.21.15.50:0/3134331257 conn(0x7fa2c80fe5b0 msgr2=0x7fa2c8104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 shutdown_connections 2024-05-10T20:57:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:09.692+0000 7fa2d0000700 1 -- 172.21.15.50:0/3134331257 wait complete. 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.814 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: Updating smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.815 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.815 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.815 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-10T20:57:09.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-10T20:57:09.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-10T20:57:09.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: Updating smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:09.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.426 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T20:57:10.427 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:57:10.427 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T20:57:10.511 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:57:10.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:57:10.614 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:57:10.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T20:57:10.645 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:57:10.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:10.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: Reconfiguring daemon mon.a on smithi050 2024-05-10T20:57:10.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:10 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3134331257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.711 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: Reconfiguring daemon mon.a on smithi050 2024-05-10T20:57:10.712 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:10 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3134331257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.714 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:57:10.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T20:57:10.741 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:10.741 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.741 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.741 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: Reconfiguring daemon mon.a on smithi050 2024-05-10T20:57:10.742 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:10 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3134331257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:10.746 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:57:10.746 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T20:57:10.814 INFO:tasks.cephadm:Adding mgr.x on smithi050 2024-05-10T20:57:10.814 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-05-10T20:57:10.814 INFO:tasks.cephadm:Adding mgr.z on smithi142 2024-05-10T20:57:10.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi050=x;smithi092=y;smithi142=z' 2024-05-10T20:57:11.085 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:57:11.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:11.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:11.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 -- 172.21.15.142:0/864624287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 msgr2=0x7f188c106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 --2- 172.21.15.142:0/864624287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 0x7f188c106c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1888009910 tx=0x7f1888009c20 comp rx=0 tx=0).stop 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 -- 172.21.15.142:0/864624287 shutdown_connections 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 --2- 172.21.15.142:0/864624287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 0x7f188c106c10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 -- 172.21.15.142:0/864624287 >> 172.21.15.142:0/864624287 conn(0x7f188c0fe250 msgr2=0x7f188c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 -- 172.21.15.142:0/864624287 shutdown_connections 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.513+0000 7f189427f700 1 -- 172.21.15.142:0/864624287 wait complete. 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 Processor -- start 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 -- start start 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 0x7f188c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 0x7f188c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:12.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f188c109cb0 con 0x7f188c1047f0 2024-05-10T20:57:12.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f188c109b30 con 0x7f188c199f60 2024-05-10T20:57:12.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189427f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f188c109e30 con 0x7f188c19f430 2024-05-10T20:57:12.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189281c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:12.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189281c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36362/0 (socket says 172.21.15.142:36362) 2024-05-10T20:57:12.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189281c700 1 -- 172.21.15.142:0/399764449 learned_addr learned my addr 172.21.15.142:0/399764449 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.514+0000 7f189181a700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 0x7f188c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 -- 172.21.15.142:0/399764449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 msgr2=0x7f188c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 0x7f188c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 -- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 msgr2=0x7f188c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 0x7f188c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 -- 172.21.15.142:0/399764449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1a1dc0 con 0x7f188c19f430 2024-05-10T20:57:12.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189181a700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 0x7f188c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T20:57:12.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189281c700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f188400ba50 tx=0x7f188400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:12.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884019070 con 0x7f188c19f430 2024-05-10T20:57:12.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1884014440 con 0x7f188c19f430 2024-05-10T20:57:12.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c1a20b0 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884007590 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.515+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1a25f0 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.516+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f188401c070 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.516+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c064e20 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.516+0000 7f18837fe700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 0x7f1878040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.516+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1884051ed0 con 0x7f188c19f430 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.517+0000 7f189201b700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 0x7f1878040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:12.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.518+0000 7f189201b700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 0x7f1878040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1888006010 tx=0x7f1888005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:12.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.522+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1884022170 con 0x7f188c19f430 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:12.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:12.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.736+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi092=y;smithi142=z", "target": ["mon-mgr", ""]}) -- 0x7f188c1000a0 con 0x7f187803e9a0 2024-05-10T20:57:12.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.752+0000 7f18837fe700 1 -- 172.21.15.142:0/399764449 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f188c1000a0 con 0x7f187803e9a0 2024-05-10T20:57:12.757 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mgr update... 2024-05-10T20:57:12.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 msgr2=0x7f1878040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:12.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 0x7f1878040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1888006010 tx=0x7f1888005dc0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 msgr2=0x7f188c1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f188400ba50 tx=0x7f188400be10 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 shutdown_connections 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f187803e9a0 0x7f1878040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f188c19f430 0x7f188c1a1850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f188c1047f0 0x7f188c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 --2- 172.21.15.142:0/399764449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c199f60 0x7f188c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 >> 172.21.15.142:0/399764449 conn(0x7f188c0fe250 msgr2=0x7f188c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 shutdown_connections 2024-05-10T20:57:12.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:57:12.754+0000 7f189427f700 1 -- 172.21.15.142:0/399764449 wait complete. 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:12.851 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:12.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:12.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:12.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:12.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:12.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:13.300 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T20:57:13.303 DEBUG:teuthology.orchestra.run.smithi142:mgr.z> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T20:57:13.306 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T20:57:13.306 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T20:57:13.306 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T20:57:13.369 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T20:57:13.369 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-05-10T20:57:13.428 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-10 20:56:09.021321422 +0000 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-10 20:50:24.589539049 +0000 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-10 20:50:24.589539049 +0000 2024-05-10T20:57:13.429 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-10 20:50:24.589539049 +0000 2024-05-10T20:57:13.429 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T20:57:13.519 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-10T20:57:13.519 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-10T20:57:13.519 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000281054 s, 1.8 MB/s 2024-05-10T20:57:13.523 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T20:57:13.582 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-10 20:56:09.096319510 +0000 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-10 20:50:25.182524256 +0000 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-10 20:50:25.182524256 +0000 2024-05-10T20:57:13.636 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-10 20:50:25.182524256 +0000 2024-05-10T20:57:13.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T20:57:13.713 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-10T20:57:13.714 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-10T20:57:13.714 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000286925 s, 1.8 MB/s 2024-05-10T20:57:13.717 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T20:57:13.773 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-10 20:56:09.178317419 +0000 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-10 20:50:25.833508017 +0000 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-10 20:50:25.833508017 +0000 2024-05-10T20:57:13.822 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-10 20:50:25.833508017 +0000 2024-05-10T20:57:13.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T20:57:13.904 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-10T20:57:13.904 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-10T20:57:13.905 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000265743 s, 1.9 MB/s 2024-05-10T20:57:13.908 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T20:57:13.969 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-05-10T20:57:14.024 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T20:57:14.024 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.024 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 692 Links: 1 2024-05-10T20:57:14.024 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.025 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.025 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-10 20:56:09.251315558 +0000 2024-05-10T20:57:14.025 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-10 20:50:26.454492527 +0000 2024-05-10T20:57:14.025 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-10 20:50:26.454492527 +0000 2024-05-10T20:57:14.025 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-10 20:50:26.454492527 +0000 2024-05-10T20:57:14.025 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T20:57:14.064 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: Reconfiguring mon.b (monmap changed)... 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: Reconfiguring daemon mon.b on smithi092 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi092=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: Saving service mgr spec with placement smithi050=x;smithi092=y;smithi142=z;count:3 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:14.065 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:14.101 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: Reconfiguring mon.b (monmap changed)... 2024-05-10T20:57:14.101 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: Reconfiguring daemon mon.b on smithi092 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi092=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: Saving service mgr spec with placement smithi050=x;smithi092=y;smithi142=z;count:3 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:14.102 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:14.112 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-10T20:57:14.112 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-10T20:57:14.112 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000265928 s, 1.9 MB/s 2024-05-10T20:57:14.118 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: Reconfiguring mon.b (monmap changed)... 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: Reconfiguring daemon mon.b on smithi092 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi092=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: Saving service mgr spec with placement smithi050=x;smithi092=y;smithi142=z;count:3 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T20:57:14.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:14.173 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-10T20:57:14.173 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T20:57:14.192 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T20:57:14.192 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-10 20:57:00.522201698 +0000 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-10 20:49:44.003801393 +0000 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-10 20:49:44.003801393 +0000 2024-05-10T20:57:14.250 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-10 20:49:44.003801393 +0000 2024-05-10T20:57:14.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T20:57:14.315 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-10T20:57:14.316 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-10T20:57:14.316 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000287396 s, 1.8 MB/s 2024-05-10T20:57:14.317 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T20:57:14.374 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-05-10T20:57:14.431 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-10 20:57:00.522201698 +0000 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-10 20:49:44.392790296 +0000 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-10 20:49:44.392790296 +0000 2024-05-10T20:57:14.432 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-10 20:49:44.392790296 +0000 2024-05-10T20:57:14.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T20:57:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-10T20:57:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-10T20:57:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000304771 s, 1.7 MB/s 2024-05-10T20:57:14.498 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T20:57:14.556 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.612 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-10 20:57:00.523201670 +0000 2024-05-10T20:57:14.613 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-10 20:49:44.745780227 +0000 2024-05-10T20:57:14.613 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-10 20:49:44.745780227 +0000 2024-05-10T20:57:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-10 20:49:44.745780227 +0000 2024-05-10T20:57:14.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T20:57:14.676 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-10T20:57:14.677 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-10T20:57:14.677 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000227139 s, 2.3 MB/s 2024-05-10T20:57:14.678 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T20:57:14.735 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-05-10T20:57:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T20:57:14.792 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-10 20:57:00.523201670 +0000 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-10 20:49:45.070770957 +0000 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-10 20:49:45.070770957 +0000 2024-05-10T20:57:14.793 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-10 20:49:45.070770957 +0000 2024-05-10T20:57:14.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T20:57:14.857 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-10T20:57:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-10T20:57:14.858 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000209333 s, 2.4 MB/s 2024-05-10T20:57:14.859 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T20:57:14.916 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-10T20:57:14.916 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T20:57:14.933 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T20:57:14.934 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 20:56:50.951112169 +0000 2024-05-10T20:57:14.991 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 20:49:11.915853939 +0000 2024-05-10T20:57:14.992 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 20:49:11.915853939 +0000 2024-05-10T20:57:14.992 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-10 20:49:11.915853939 +0000 2024-05-10T20:57:14.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T20:57:15.053 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:14 smithi142 ceph-mon[34846]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:15.053 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:14 smithi142 ceph-mon[34846]: Reconfiguring mon.c (monmap changed)... 2024-05-10T20:57:15.054 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:14 smithi142 ceph-mon[34846]: Reconfiguring daemon mon.c on smithi142 2024-05-10T20:57:15.056 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T20:57:15.056 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T20:57:15.056 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000193337 s, 2.6 MB/s 2024-05-10T20:57:15.058 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T20:57:15.115 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-05-10T20:57:15.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:14 smithi092 ceph-mon[34415]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:15.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:14 smithi092 ceph-mon[34415]: Reconfiguring mon.c (monmap changed)... 2024-05-10T20:57:15.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:14 smithi092 ceph-mon[34415]: Reconfiguring daemon mon.c on smithi142 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 20:56:50.952112150 +0000 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 20:49:12.253847547 +0000 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 20:49:12.253847547 +0000 2024-05-10T20:57:15.172 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-10 20:49:12.253847547 +0000 2024-05-10T20:57:15.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T20:57:15.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:14 smithi050 ceph-mon[29546]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:15.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:14 smithi050 ceph-mon[29546]: Reconfiguring mon.c (monmap changed)... 2024-05-10T20:57:15.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:14 smithi050 ceph-mon[29546]: Reconfiguring daemon mon.c on smithi142 2024-05-10T20:57:15.235 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T20:57:15.236 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T20:57:15.236 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000199325 s, 2.6 MB/s 2024-05-10T20:57:15.237 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T20:57:15.294 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 20:56:50.952112150 +0000 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 20:49:12.584841289 +0000 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 20:49:12.584841289 +0000 2024-05-10T20:57:15.349 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-10 20:49:12.584841289 +0000 2024-05-10T20:57:15.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T20:57:15.411 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T20:57:15.411 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T20:57:15.411 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000237126 s, 2.2 MB/s 2024-05-10T20:57:15.412 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T20:57:15.468 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-05-10T20:57:15.524 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T20:57:15.524 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T20:57:15.524 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-10 20:56:50.953112132 +0000 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-10 20:49:12.919834954 +0000 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-10 20:49:12.919834954 +0000 2024-05-10T20:57:15.525 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-10 20:49:12.919834954 +0000 2024-05-10T20:57:15.526 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T20:57:15.591 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-10T20:57:15.591 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-10T20:57:15.591 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000187525 s, 2.7 MB/s 2024-05-10T20:57:15.592 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T20:57:15.647 INFO:tasks.cephadm:Deploying osd.0 on smithi050 with /dev/vg_nvme/lv_4... 2024-05-10T20:57:15.648 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T20:57:16.101 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:16.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: Deploying daemon mgr.z on smithi142 2024-05-10T20:57:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:15 smithi050 ceph-mon[29546]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: Deploying daemon mgr.z on smithi142 2024-05-10T20:57:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:15 smithi092 ceph-mon[34415]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:16.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:16.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:16.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: Deploying daemon mgr.z on smithi142 2024-05-10T20:57:16.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:15 smithi142 ceph-mon[34846]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:17.316 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 systemd[1]: Starting Ceph mgr.z for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T20:57:17.581 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 podman[36690]: 2024-05-10 20:57:17.214081201 +0000 UTC m=+0.017854433 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T20:57:17.581 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 podman[36690]: 2024-05-10 20:57:17.316004508 +0000 UTC m=+0.119777731 container create f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T20:57:17.581 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 podman[36690]: 2024-05-10 20:57:17.5766495 +0000 UTC m=+0.380422723 container init f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T20:57:17.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 podman[36690]: 2024-05-10 20:57:17.579501898 +0000 UTC m=+0.383275127 container start f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-10T20:57:17.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 ceph-mgr[36709]: -- 172.21.15.142:0/4014048499 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563440a1c5a0 con 0x5634457a6900 2024-05-10T20:57:17.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 bash[36690]: f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 2024-05-10T20:57:17.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 systemd[1]: Started Ceph mgr.z for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:57:17.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:17.745+0000 7f721db29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T20:57:18.348 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:17.932+0000 7f721db29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:57:18.348 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:18.034+0000 7f721db29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T20:57:18.348 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:18.343+0000 7f721db29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T20:57:18.635 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:18.427+0000 7f721db29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:19.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0382694 s, 274 MB/s 2024-05-10T20:57:19.158 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-10T20:57:19.158 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:19.158 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:19.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:19.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:19 smithi092 ceph-mon[34415]: Deploying daemon mgr.y on smithi092 2024-05-10T20:57:19.911 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:19 smithi050 ceph-mon[29546]: Deploying daemon mgr.y on smithi092 2024-05-10T20:57:19.970 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-05-10T20:57:19.998 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:19 smithi142 ceph-mon[34846]: Deploying daemon mgr.y on smithi092 2024-05-10T20:57:19.998 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:19.640+0000 7f721db29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T20:57:19.998 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:19.815+0000 7f721db29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T20:57:19.999 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:19.993+0000 7f721db29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T20:57:20.239 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mgr[35800]: -- 172.21.15.92:0/1653601261 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c0d9cc5a0 con 0x557c12756900 2024-05-10T20:57:20.239 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 bash[35780]: 8eef36b1a8201205fb4ec92a8ed4926ee55c89c566fcf93862dee268cd1937e3 2024-05-10T20:57:20.429 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:57:20.494 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 systemd[1]: Started Ceph mgr.y for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T20:57:20.494 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:20.302+0000 7ffb19d54200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T20:57:20.589 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:20.284+0000 7f721db29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T20:57:20.589 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:20.377+0000 7f721db29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T20:57:20.589 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:20.583+0000 7f721db29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T20:57:20.777 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:20.497+0000 7ffb19d54200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:57:20.777 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:20.605+0000 7ffb19d54200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:20.784+0000 7f721db29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:20.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:20.933+0000 7ffb19d54200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:21.022+0000 7ffb19d54200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:21.028 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:21.700 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:21.429+0000 7f721db29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T20:57:21.700 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:21.661+0000 7f721db29200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T20:57:22.385 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:22.094+0000 7f721db29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T20:57:22.385 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:22.190+0000 7f721db29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T20:57:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.537+0000 7f592d000700 1 -- 172.21.15.50:0/4168989027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928078770 msgr2=0x7f5928076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.537+0000 7f592d000700 1 --2- 172.21.15.50:0/4168989027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928078770 0x7f5928076c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5918009990 tx=0x7f5918009ca0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 -- 172.21.15.50:0/4168989027 shutdown_connections 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 --2- 172.21.15.50:0/4168989027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f592810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 --2- 172.21.15.50:0/4168989027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928078770 0x7f5928076c40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 --2- 172.21.15.50:0/4168989027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928077180 0x7f5928077590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 -- 172.21.15.50:0/4168989027 >> 172.21.15.50:0/4168989027 conn(0x7f59280ff9d0 msgr2=0x7f5928101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 -- 172.21.15.50:0/4168989027 shutdown_connections 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.538+0000 7f592d000700 1 -- 172.21.15.50:0/4168989027 wait complete. 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.539+0000 7f592d000700 1 Processor -- start 2024-05-10T20:57:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.539+0000 7f592d000700 1 -- start start 2024-05-10T20:57:22.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.539+0000 7f592d000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 0x7f592819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:22.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f592d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928078770 0x7f592819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:22.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f592d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f59281a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:22.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f592d000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5928112090 con 0x7f5928077180 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f592d000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5928111f10 con 0x7f5928078770 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f592d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5928112210 con 0x7f592810c250 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5925e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 0x7f592819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5925400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928078770 0x7f592819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5925e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 0x7f592819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41424/0 (socket says 172.21.15.50:41424) 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5925e00700 1 -- 172.21.15.50:0/1898822060 learned_addr learned my addr 172.21.15.50:0/1898822060 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:57:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f59281a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 -- 172.21.15.50:0/1898822060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928078770 msgr2=0x7f592819d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928078770 0x7f592819d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 -- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 msgr2=0x7f592819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 0x7f592819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.540+0000 7f5926800700 1 -- 172.21.15.50:0/1898822060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59281a4730 con 0x7f592810c250 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f5926800700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f59281a40b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f592000efc0 tx=0x7f592000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59200092e0 con 0x7f592810c250 2024-05-10T20:57:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920004bb0 con 0x7f592810c250 2024-05-10T20:57:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920004020 con 0x7f592810c250 2024-05-10T20:57:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281a49c0 con 0x7f592810c250 2024-05-10T20:57:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.541+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281a4eb0 con 0x7f592810c250 2024-05-10T20:57:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.543+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280512c0 con 0x7f592810c250 2024-05-10T20:57:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.543+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f592001b030 con 0x7f592810c250 2024-05-10T20:57:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.543+0000 7f591ea00700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 0x7f590c040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.544+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f59200525c0 con 0x7f592810c250 2024-05-10T20:57:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.544+0000 7f5925e00700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 0x7f590c040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.545+0000 7f5925e00700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 0x7f590c040c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f591800b2e0 tx=0x7f5918009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:22.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.549+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5920022020 con 0x7f592810c250 2024-05-10T20:57:22.614 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:22.272+0000 7ffb19d54200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T20:57:22.614 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:22.440+0000 7ffb19d54200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T20:57:22.614 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:22.609+0000 7ffb19d54200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T20:57:22.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:22.802+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5928063c00 con 0x7f590c03e750 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:22.808 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.809 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:22.809 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:22.809 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:22.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:22.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:22.561+0000 7f721db29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T20:57:22.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T20:57:22.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:23.186 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:22.891+0000 7ffb19d54200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T20:57:23.187 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:22.982+0000 7ffb19d54200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T20:57:23.187 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:23.181+0000 7ffb19d54200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T20:57:23.243 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:22.911+0000 7f721db29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T20:57:23.244 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.002+0000 7f721db29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T20:57:23.624 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:23.370+0000 7ffb19d54200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T20:57:23.635 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.238+0000 7f721db29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T20:57:23.635 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.368+0000 7f721db29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T20:57:23.635 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.461+0000 7f721db29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T20:57:24.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:23 smithi050 ceph-mon[29546]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T20:57:24.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:23 smithi050 ceph-mon[29546]: Reconfiguring daemon mgr.x on smithi050 2024-05-10T20:57:24.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:24.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:24.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:24.124 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:23.959+0000 7ffb19d54200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T20:57:24.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:23 smithi092 ceph-mon[34415]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T20:57:24.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:23 smithi092 ceph-mon[34415]: Reconfiguring daemon mgr.x on smithi050 2024-05-10T20:57:24.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:24.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:24.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.787+0000 7f721db29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:23.878+0000 7f721db29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:23 smithi142 ceph-mon[34846]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:23 smithi142 ceph-mon[34846]: Reconfiguring daemon mgr.x on smithi050 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:24.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:24.598 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:24.178+0000 7ffb19d54200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T20:57:24.793 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:24.466+0000 7f721db29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T20:57:24.793 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 20:57:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z[36705]: 2024-05-10T20:57:24.680+0000 7f721db29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T20:57:24.874 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:24.593+0000 7ffb19d54200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T20:57:24.874 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:24.688+0000 7ffb19d54200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T20:57:24.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: Standby manager daemon z started 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T20:57:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:24 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: Standby manager daemon z started 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T20:57:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:24 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:25.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: Standby manager daemon z started 2024-05-10T20:57:25.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T20:57:25.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:25.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T20:57:25.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:24 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.142:0/1153743540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:25.374 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.056+0000 7ffb19d54200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T20:57:25.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:25.592+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f5920012050 con 0x7f592810c250 2024-05-10T20:57:25.740 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.404+0000 7ffb19d54200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T20:57:25.740 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.494+0000 7ffb19d54200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T20:57:26.124 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.735+0000 7ffb19d54200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T20:57:26.124 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.868+0000 7ffb19d54200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T20:57:26.124 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:25.962+0000 7ffb19d54200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T20:57:26.595 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:26.283+0000 7ffb19d54200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T20:57:26.595 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:26.374+0000 7ffb19d54200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T20:57:26.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:26 smithi092 ceph-mon[34415]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:26.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:26 smithi092 ceph-mon[34415]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-10T20:57:26.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:26 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T20:57:26.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:26 smithi142 ceph-mon[34846]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:26.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:26 smithi142 ceph-mon[34846]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-10T20:57:26.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:26 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T20:57:26.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:26 smithi050 ceph-mon[29546]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:26.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:26 smithi050 ceph-mon[29546]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-10T20:57:26.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:26 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T20:57:27.376 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:26.961+0000 7ffb19d54200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T20:57:27.376 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 20:57:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y[35795]: 2024-05-10T20:57:27.174+0000 7ffb19d54200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T20:57:27.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:27.618+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mon.1 v2:172.21.15.142:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f592001cc90 con 0x7f592810c250 2024-05-10T20:57:27.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:27 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T20:57:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:27 smithi092 ceph-mon[34415]: Standby manager daemon y started 2024-05-10T20:57:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:27 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:27 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T20:57:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:27 smithi092 ceph-mon[34415]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:27 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T20:57:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:27 smithi142 ceph-mon[34846]: Standby manager daemon y started 2024-05-10T20:57:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:27 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:27 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T20:57:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:27 smithi142 ceph-mon[34846]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:27.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:27 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T20:57:27.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:27 smithi050 ceph-mon[29546]: Standby manager daemon y started 2024-05-10T20:57:27.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:27 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T20:57:27.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:27 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T20:57:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:27 smithi050 ceph-mon[29546]: from='mgr.? 172.21.15.92:0/619247729' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]: dispatch 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]': finished 2024-05-10T20:57:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T20:57:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]: dispatch 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]': finished 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T20:57:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:28.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:28.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-10T20:57:28.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T20:57:28.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]: dispatch 2024-05-10T20:57:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1524974811' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87ed2f28-e9c4-4c8e-a444-06e540010639"}]': finished 2024-05-10T20:57:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T20:57:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:29.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:29 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2744595887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:57:29.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:29 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2744595887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:57:30.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:29 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2744595887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:57:30.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:30 smithi142 ceph-mon[34846]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:30.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:30 smithi050 ceph-mon[29546]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:31.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:30 smithi092 ceph-mon[34415]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:32.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:32 smithi142 ceph-mon[34846]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:32.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:32 smithi050 ceph-mon[29546]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:33.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:32 smithi092 ceph-mon[34415]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:34.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:34 smithi050 ceph-mon[29546]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:35.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:34 smithi092 ceph-mon[34415]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:34 smithi142 ceph-mon[34846]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:36.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:36 smithi050 ceph-mon[29546]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:37.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:36 smithi092 ceph-mon[34415]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:37.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:36 smithi142 ceph-mon[34846]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:37.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:37 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T20:57:37.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:37 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:38.036 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:37 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T20:57:38.036 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:37 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:38.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:37 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T20:57:38.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:37 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:38.754 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:38 smithi050 ceph-mon[29546]: Deploying daemon osd.0 on smithi050 2024-05-10T20:57:38.754 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:38 smithi050 ceph-mon[29546]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:39.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:38 smithi092 ceph-mon[34415]: Deploying daemon osd.0 on smithi050 2024-05-10T20:57:39.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:38 smithi092 ceph-mon[34415]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:39.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:38 smithi142 ceph-mon[34846]: Deploying daemon osd.0 on smithi050 2024-05-10T20:57:39.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:38 smithi142 ceph-mon[34846]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:40 smithi050 ceph-mon[29546]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:41.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:40 smithi092 ceph-mon[34415]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:41.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:40 smithi142 ceph-mon[34846]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:42.844 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:42 smithi050 ceph-mon[29546]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:43.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:42 smithi092 ceph-mon[34415]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:43.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:42 smithi142 ceph-mon[34846]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:44.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:44 smithi050 ceph-mon[29546]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:45.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:44 smithi092 ceph-mon[34415]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:45.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:44 smithi142 ceph-mon[34846]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:46.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:46 smithi050 ceph-mon[29546]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:46.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:46.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:46 smithi092 ceph-mon[34415]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:46 smithi142 ceph-mon[34846]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:47.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:48.025 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:48.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:48.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:48 smithi050 ceph-mon[29546]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:49.040 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:48 smithi092 ceph-mon[34415]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:49.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:48 smithi142 ceph-mon[34846]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:51.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:51.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:51.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:51.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:51 smithi050 ceph-mon[29546]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T20:57:52.113 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 0 on host 'smithi050' 2024-05-10T20:57:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.106+0000 7f591ea00700 1 -- 172.21.15.50:0/1898822060 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5928063c00 con 0x7f590c03e750 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 msgr2=0x7f590c040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 0x7f590c040c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f591800b2e0 tx=0x7f5918009590 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 msgr2=0x7f59281a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f59281a40b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f592000efc0 tx=0x7f592000c3a0 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 shutdown_connections 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f590c03e750 0x7f590c040c60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f592810c250 0x7f59281a40b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5928077180 0x7f592819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 --2- 172.21.15.50:0/1898822060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5928078770 0x7f592819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.110+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 >> 172.21.15.50:0/1898822060 conn(0x7f59280ff9d0 msgr2=0x7f5928108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.111+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 shutdown_connections 2024-05-10T20:57:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:57:52.111+0000 7f592d000700 1 -- 172.21.15.50:0/1898822060 wait complete. 2024-05-10T20:57:52.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:51 smithi092 ceph-mon[34415]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T20:57:52.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:51 smithi142 ceph-mon[34846]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:52.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:52.934 DEBUG:teuthology.orchestra.run.smithi050:osd.0> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T20:57:52.939 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-05-10T20:57:52.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T20:57:52.968 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:52.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T20:57:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T20:57:53.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:57:53.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:53.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:53.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:53.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:57:53.187 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T20:57:53.997 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:53 smithi050 ceph-mon[29546]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:57:53.997 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:53 smithi050 ceph-mon[29546]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T20:57:53.997 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:53.997 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:53 smithi092 ceph-mon[34415]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:57:54.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:53 smithi092 ceph-mon[34415]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T20:57:54.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:53 smithi142 ceph-mon[34846]: from='osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:57:54.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:53 smithi142 ceph-mon[34846]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T20:57:54.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] boot 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T20:57:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] boot 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T20:57:54.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01138 s, 921 MB/s 2024-05-10T20:57:55.063 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: osd.0 [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] boot 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T20:57:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T20:57:55.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-05-10T20:57:56.043 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T20:57:56.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:55 smithi092 ceph-mon[34415]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T20:57:56.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:55 smithi142 ceph-mon[34846]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T20:57:56.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:55 smithi050 ceph-mon[29546]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T20:57:57.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:56 smithi050 ceph-mon[29546]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:57.113 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:56 smithi092 ceph-mon[34415]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:57.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:56 smithi142 ceph-mon[34846]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 -- 172.21.15.92:0/1482232589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84301059b0 msgr2=0x7f8430105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 --2- 172.21.15.92:0/1482232589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84301059b0 0x7f8430105dc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f842c009990 tx=0x7f842c009ca0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 -- 172.21.15.92:0/1482232589 shutdown_connections 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 --2- 172.21.15.92:0/1482232589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84301059b0 0x7f8430105dc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 --2- 172.21.15.92:0/1482232589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84301049e0 0x7f843010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 --2- 172.21.15.92:0/1482232589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8430103f60 0x7f8430104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 -- 172.21.15.92:0/1482232589 >> 172.21.15.92:0/1482232589 conn(0x7f84300ff9d0 msgr2=0x7f8430101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 -- 172.21.15.92:0/1482232589 shutdown_connections 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.624+0000 7f84384f0700 1 -- 172.21.15.92:0/1482232589 wait complete. 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 Processor -- start 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 -- start start 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84301049e0 0x7f843019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84301059b0 0x7f84301a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f843010ffa0 con 0x7f84301049e0 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f843010fe20 con 0x7f8430103f60 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f84384f0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8430110120 con 0x7f84301059b0 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f843628c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f843628c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33950/0 (socket says 172.21.15.92:33950) 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.625+0000 7f843628c700 1 -- 172.21.15.92:0/860816155 learned_addr learned my addr 172.21.15.92:0/860816155 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f8436a8d700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84301059b0 0x7f84301a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 -- 172.21.15.92:0/860816155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84301059b0 msgr2=0x7f84301a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84301059b0 0x7f84301a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 -- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84301049e0 msgr2=0x7f843019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:57:57.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84301049e0 0x7f843019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 -- 172.21.15.92:0/860816155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301a2610 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f843628c700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098e0 tx=0x7f842c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c027070 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c022470 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c00f6c0 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84301a28a0 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.626+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301a2dd0 con 0x7f8430103f60 2024-05-10T20:57:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.627+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84300512c0 con 0x7f8430103f60 2024-05-10T20:57:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.628+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f842c02a070 con 0x7f8430103f60 2024-05-10T20:57:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.629+0000 7f84277fe700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 0x7f841c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:57:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.630+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f842c0e1a70 con 0x7f8430103f60 2024-05-10T20:57:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.630+0000 7f8435a8b700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 0x7f841c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:57:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.631+0000 7f8435a8b700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 0x7f841c0b4d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f843019bb60 tx=0x7f8420005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:57:57.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.634+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f842c013070 con 0x7f8430103f60 2024-05-10T20:57:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:57:57.845+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f843002ea00 con 0x7f841c0b2870 2024-05-10T20:57:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:57:59.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:57:59.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:00.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:59 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi050 2024-05-10T20:58:00.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:59 smithi092 ceph-mon[34415]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:00.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:57:59 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi050 to 17094M 2024-05-10T20:58:00.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:59 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi050 2024-05-10T20:58:00.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:59 smithi142 ceph-mon[34846]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:00.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:57:59 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi050 to 17094M 2024-05-10T20:58:00.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:59 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi050 2024-05-10T20:58:00.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:59 smithi050 ceph-mon[29546]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:00.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:57:59 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi050 to 17094M 2024-05-10T20:58:01.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:01.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/1197095598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]': finished 2024-05-10T20:58:01.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T20:58:01.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:00 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:01.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/1197095598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]': finished 2024-05-10T20:58:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T20:58:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:00 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/1197095598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]: dispatch 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afef3a38-83d4-4d62-9db8-065d439b4fda"}]': finished 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T20:58:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:00 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:02.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:01 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/1034400513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:02.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:01 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/1034400513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:02.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:01 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/1034400513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:03.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:02 smithi092 ceph-mon[34415]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:02 smithi142 ceph-mon[34846]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:03.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:02 smithi050 ceph-mon[29546]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:05.038 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:04 smithi092 ceph-mon[34415]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:05.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:04 smithi142 ceph-mon[34846]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:05.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:04 smithi050 ceph-mon[29546]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:06.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:06 smithi092 ceph-mon[34415]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:07.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:06 smithi142 ceph-mon[34846]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:07.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:06 smithi050 ceph-mon[29546]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:08.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:07 smithi092 ceph-mon[34415]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:08.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:07 smithi142 ceph-mon[34846]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:08.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:07 smithi050 ceph-mon[29546]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:09.081 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T20:58:09.081 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:09.081 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:08 smithi092 ceph-mon[34415]: Deploying daemon osd.1 on smithi092 2024-05-10T20:58:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T20:58:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:08 smithi142 ceph-mon[34846]: Deploying daemon osd.1 on smithi092 2024-05-10T20:58:09.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T20:58:09.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:09.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:08 smithi050 ceph-mon[29546]: Deploying daemon osd.1 on smithi092 2024-05-10T20:58:10.054 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:09 smithi092 ceph-mon[34415]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:10.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:09 smithi142 ceph-mon[34846]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:10.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:09 smithi050 ceph-mon[29546]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:12.729 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:12 smithi092 ceph-mon[34415]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:12.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:12 smithi142 ceph-mon[34846]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:12.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:12 smithi050 ceph-mon[29546]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:14.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:14.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:14 smithi092 ceph-mon[34415]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:14.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:14.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:14 smithi142 ceph-mon[34846]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:14.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:14.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:14.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:14 smithi050 ceph-mon[29546]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:16.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:16.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:16.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:16.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:16 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.758 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:16.758 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.758 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.759 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:16.759 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:16.759 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:16 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:16.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:16 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:17.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.087+0000 7f84277fe700 1 -- 172.21.15.92:0/860816155 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f843002ea00 con 0x7f841c0b2870 2024-05-10T20:58:17.092 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 msgr2=0x7f841c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 0x7f841c0b4d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f843019bb60 tx=0x7f8420005c80 comp rx=0 tx=0).stop 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 msgr2=0x7f843019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098e0 tx=0x7f842c0047b0 comp rx=0 tx=0).stop 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 shutdown_connections 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f841c0b2870 0x7f841c0b4d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8430103f60 0x7f843019aab0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.088+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84301059b0 0x7f84301a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.089+0000 7f84384f0700 1 --2- 172.21.15.92:0/860816155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84301049e0 0x7f843019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.089+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 >> 172.21.15.92:0/860816155 conn(0x7f84300ff9d0 msgr2=0x7f84301086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:58:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.089+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 shutdown_connections 2024-05-10T20:58:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:58:17.089+0000 7f84384f0700 1 -- 172.21.15.92:0/860816155 wait complete. 2024-05-10T20:58:17.684 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T20:58:17.686 INFO:tasks.cephadm:Deploying osd.2 on smithi142 with /dev/vg_nvme/lv_4... 2024-05-10T20:58:17.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T20:58:17.928 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:58:18.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:18.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:18 smithi092 ceph-mon[34415]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:18 smithi142 ceph-mon[34846]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:18.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:18.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:18.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T20:58:18.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:18 smithi050 ceph-mon[29546]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:19.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:19 smithi092 ceph-mon[34415]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T20:58:19.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:19 smithi092 ceph-mon[34415]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:19.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:19 smithi092 ceph-mon[34415]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T20:58:19.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:19.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:19 smithi092 ceph-mon[34415]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:19.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:19 smithi142 ceph-mon[34846]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T20:58:19.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:19 smithi142 ceph-mon[34846]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:19.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:19 smithi142 ceph-mon[34846]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T20:58:19.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:19.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:19 smithi142 ceph-mon[34846]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:19.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:19 smithi050 ceph-mon[29546]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T20:58:19.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:19 smithi050 ceph-mon[29546]: from='osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:19.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:19 smithi050 ceph-mon[29546]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T20:58:19.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:19.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:19 smithi050 ceph-mon[29546]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112999 s, 928 MB/s 2024-05-10T20:58:20.067 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T20:58:20.167 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:58:20.167 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T20:58:20.167 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.168 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.168 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:20.168 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:20 smithi092 ceph-mon[34415]: from='osd.1 ' entity='osd.1' 2024-05-10T20:58:20.168 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1[39012]: 2024-05-10T20:58:20.022+0000 7fd2421d4700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T20:58:20.168 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1[39012]: 2024-05-10T20:58:20.161+0000 7fd239780700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:20.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:20 smithi142 ceph-mon[34846]: from='osd.1 ' entity='osd.1' 2024-05-10T20:58:20.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:58:20.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T20:58:20.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:20.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:20.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:20 smithi050 ceph-mon[29546]: from='osd.1 ' entity='osd.1' 2024-05-10T20:58:20.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-05-10T20:58:20.944 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi092 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:21.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-10T20:58:21.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:21.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:21.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi092 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-10T20:58:21.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:21.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:21.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:21.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T20:58:21.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi092 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:21.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:22.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:22 smithi142 ceph-mon[34846]: osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] boot 2024-05-10T20:58:22.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:22 smithi142 ceph-mon[34846]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T20:58:22.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:22.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:22 smithi142 ceph-mon[34846]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:22.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:22 smithi050 ceph-mon[29546]: osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] boot 2024-05-10T20:58:22.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:22 smithi050 ceph-mon[29546]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T20:58:22.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:22.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:22 smithi050 ceph-mon[29546]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 -- 172.21.15.142:0/2569573271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0104370 msgr2=0x7f61d0104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 --2- 172.21.15.142:0/2569573271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0104370 0x7f61d0104780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009990 tx=0x7f61c4009ca0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 -- 172.21.15.142:0/2569573271 shutdown_connections 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 --2- 172.21.15.142:0/2569573271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0105e40 0x7f61d010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 --2- 172.21.15.142:0/2569573271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0104370 0x7f61d0104780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 --2- 172.21.15.142:0/2569573271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d01054a0 0x7f61d01058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 -- 172.21.15.142:0/2569573271 >> 172.21.15.142:0/2569573271 conn(0x7f61d00ff9d0 msgr2=0x7f61d0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:58:22.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.496+0000 7f61d6d93700 1 -- 172.21.15.142:0/2569573271 shutdown_connections 2024-05-10T20:58:22.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.497+0000 7f61d6d93700 1 -- 172.21.15.142:0/2569573271 wait complete. 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.497+0000 7f61d6d93700 1 Processor -- start 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.497+0000 7f61d6d93700 1 -- start start 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0104370 0x7f61d019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d01054a0 0x7f61d019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61d010fe90 con 0x7f61d01054a0 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f61d010fd10 con 0x7f61d0104370 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0110010 con 0x7f61d0105e40 2024-05-10T20:58:22.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33434/0 (socket says 172.21.15.142:33434) 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 -- 172.21.15.142:0/152223234 learned_addr learned my addr 172.21.15.142:0/152223234 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d4b2f700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0104370 0x7f61d019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 -- 172.21.15.142:0/152223234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0104370 msgr2=0x7f61d019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0104370 0x7f61d019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 -- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d01054a0 msgr2=0x7f61d019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d01054a0 0x7f61d019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 -- 172.21.15.142:0/152223234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d01a2560 con 0x7f61d0105e40 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d5330700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61c800ada0 tx=0x7f61c800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8017020 con 0x7f61d0105e40 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d01a2850 con 0x7f61d0105e40 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c800ce30 con 0x7f61d0105e40 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.498+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8005350 con 0x7f61d0105e40 2024-05-10T20:58:22.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.499+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a2dc0 con 0x7f61d0105e40 2024-05-10T20:58:22.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.499+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d00512c0 con 0x7f61d0105e40 2024-05-10T20:58:22.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.501+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f61c800c980 con 0x7f61d0105e40 2024-05-10T20:58:22.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.502+0000 7f61cdffb700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 0x7f61c00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:22.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.502+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f61c80e0750 con 0x7f61d0105e40 2024-05-10T20:58:22.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.502+0000 7f61d4b2f700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 0x7f61c00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:22.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.503+0000 7f61d4b2f700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 0x7f61c00b4d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61c40038a0 tx=0x7f61c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:58:22.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.506+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61c8021020 con 0x7f61d0105e40 2024-05-10T20:58:22.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:22 smithi092 ceph-mon[34415]: osd.1 [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] boot 2024-05-10T20:58:22.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:22 smithi092 ceph-mon[34415]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T20:58:22.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T20:58:22.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:22 smithi092 ceph-mon[34415]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T20:58:22.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:22.717+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f61d0063cc0 con 0x7f61c00b27f0 2024-05-10T20:58:23.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:23 smithi050 ceph-mon[29546]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T20:58:23.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:23 smithi050 ceph-mon[29546]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:23.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:23.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:23.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:23 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:23.448 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:23 smithi142 ceph-mon[34846]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T20:58:23.449 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:23 smithi142 ceph-mon[34846]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:23.449 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:23.449 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:23.449 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:23 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:23.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:23 smithi092 ceph-mon[34415]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T20:58:23.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:23 smithi092 ceph-mon[34415]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:23.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:23.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:23.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:23 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:24.372 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:24 smithi142 ceph-mon[34846]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:24.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:24 smithi050 ceph-mon[29546]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:24.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:24 smithi092 ceph-mon[34415]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:25.559 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:25 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/1757093864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.559 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:25 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.559 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:25 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]': finished 2024-05-10T20:58:25.559 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:25 smithi142 ceph-mon[34846]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T20:58:25.559 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:25.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:25 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/1757093864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:25 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:25 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]': finished 2024-05-10T20:58:25.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:25 smithi092 ceph-mon[34415]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T20:58:25.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:25.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:25 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/1757093864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:25 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]: dispatch 2024-05-10T20:58:25.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:25 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00cad0c-573c-4648-9dee-c63912cde7a1"}]': finished 2024-05-10T20:58:25.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:25 smithi050 ceph-mon[29546]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T20:58:25.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:26.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:26 smithi092 ceph-mon[34415]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:26.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:26 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/151719622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:26.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:26 smithi142 ceph-mon[34846]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:26.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:26 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/151719622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:26 smithi050 ceph-mon[29546]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:26.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:26 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/151719622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:28 smithi092 ceph-mon[34415]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:28 smithi142 ceph-mon[34846]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:28.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:28 smithi050 ceph-mon[29546]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:30.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:30 smithi142 ceph-mon[34846]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:30.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:30 smithi092 ceph-mon[34415]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:30.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:30 smithi050 ceph-mon[29546]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:32.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:32 smithi142 ceph-mon[34846]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:32.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:32 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T20:58:32.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:32 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:32.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:32 smithi092 ceph-mon[34415]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:32.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:32 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T20:58:32.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:32 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:32.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:32 smithi050 ceph-mon[29546]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:32.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:32 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T20:58:32.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:32 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:33.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:33 smithi142 ceph-mon[34846]: Deploying daemon osd.2 on smithi142 2024-05-10T20:58:33.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:33 smithi092 ceph-mon[34415]: Deploying daemon osd.2 on smithi142 2024-05-10T20:58:33.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:33 smithi050 ceph-mon[29546]: Deploying daemon osd.2 on smithi142 2024-05-10T20:58:34.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:34 smithi092 ceph-mon[34415]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:34.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:34 smithi142 ceph-mon[34846]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:34.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:34 smithi050 ceph-mon[29546]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:36.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:36 smithi092 ceph-mon[34415]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:36.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:36 smithi142 ceph-mon[34846]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:36.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:36 smithi050 ceph-mon[29546]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:38.863 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:38 smithi142 ceph-mon[34846]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:38.863 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:38.864 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:38.864 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:38.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:38 smithi092 ceph-mon[34415]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:38.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:38.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:38.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:38.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:38 smithi050 ceph-mon[29546]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:38.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:38.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:38.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:41.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:41.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:41.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:41.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:41.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.556+0000 7f61cdffb700 1 -- 172.21.15.142:0/152223234 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61d0063cc0 con 0x7f61c00b27f0 2024-05-10T20:58:41.562 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 2 on host 'smithi142' 2024-05-10T20:58:41.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 msgr2=0x7f61c00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:41.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 0x7f61c00b4d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61c40038a0 tx=0x7f61c401a040 comp rx=0 tx=0).stop 2024-05-10T20:58:41.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 msgr2=0x7f61d01a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61c800ada0 tx=0x7f61c800c330 comp rx=0 tx=0).stop 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 shutdown_connections 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f61c00b27f0 0x7f61c00b4d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61d0104370 0x7f61d019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d0105e40 0x7f61d01a1ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 --2- 172.21.15.142:0/152223234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d01054a0 0x7f61d019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 >> 172.21.15.142:0/152223234 conn(0x7f61d00ff9d0 msgr2=0x7f61d01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 shutdown_connections 2024-05-10T20:58:41.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:58:41.559+0000 7f61d6d93700 1 -- 172.21.15.142:0/152223234 wait complete. 2024-05-10T20:58:42.115 DEBUG:teuthology.orchestra.run.smithi142:osd.2> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T20:58:42.118 INFO:tasks.cephadm:Deploying osd.3 on smithi050 with /dev/vg_nvme/lv_3... 2024-05-10T20:58:42.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T20:58:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:58:42.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:42.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:42.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:42.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:42 smithi142 ceph-mon[34846]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:42 smithi092 ceph-mon[34415]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:42.877 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:42.877 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:58:42.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:42.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:42.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:42 smithi050 ceph-mon[29546]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T20:58:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:43 smithi050 ceph-mon[29546]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T20:58:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:43 smithi050 ceph-mon[29546]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:43 smithi050 ceph-mon[29546]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T20:58:43.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:43.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:43 smithi050 ceph-mon[29546]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:44.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:43 smithi092 ceph-mon[34415]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T20:58:44.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:43 smithi092 ceph-mon[34415]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:44.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:43 smithi092 ceph-mon[34415]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T20:58:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:43 smithi092 ceph-mon[34415]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:44.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:43 smithi142 ceph-mon[34846]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T20:58:44.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:43 smithi142 ceph-mon[34846]: from='osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:44.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:43 smithi142 ceph-mon[34846]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T20:58:44.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:44.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:43 smithi142 ceph-mon[34846]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T20:58:44.136 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2[39998]: 2024-05-10T20:58:43.768+0000 7f98e9f7c700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T20:58:44.136 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2[39998]: 2024-05-10T20:58:43.772+0000 7f98e5d31700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T20:58:44.808 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:44 smithi142 ceph-mon[34846]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:44.808 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:44 smithi142 ceph-mon[34846]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T20:58:44.809 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:44 smithi142 ceph-mon[34846]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T20:58:44.809 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:45.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:44 smithi050 ceph-mon[29546]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:45.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:44 smithi050 ceph-mon[29546]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T20:58:45.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:44 smithi050 ceph-mon[29546]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T20:58:45.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:45.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:44 smithi092 ceph-mon[34415]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:45.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:44 smithi092 ceph-mon[34415]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T20:58:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:44 smithi092 ceph-mon[34415]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T20:58:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:45.605 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T20:58:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T20:58:45.606 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-10T20:58:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-10T20:58:45.606 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0331066 s, 317 MB/s 2024-05-10T20:58:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-10T20:58:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T20:58:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T20:58:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] boot 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] boot 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:46.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T20:58:46.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T20:58:46.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: osd.2 [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] boot 2024-05-10T20:58:46.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:46.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:46.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-05-10T20:58:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T20:58:47.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi142 2024-05-10T20:58:47.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-10T20:58:47.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:47.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:47.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T20:58:47.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi142 2024-05-10T20:58:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-10T20:58:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:47.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T20:58:47.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi142 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T20:58:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46587]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46587]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46590]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.298 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46590]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42873]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42873]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42873]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42873]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42876]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.477 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42876]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.478 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42876]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.478 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42876]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44034]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44034]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44034]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44037]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44037]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.637 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44028]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44028]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44028]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44031]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.638 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 20:58:47 smithi142 sudo[44031]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46597]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46597]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46600]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46600]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46600]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 sudo[46600]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42879]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42879]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42879]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42879]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T20:58:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 sudo[42882]: pam_unix(sudo:session): session closed for user root 2024-05-10T20:58:48.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: pgmap v96: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:47 smithi092 ceph-mon[34415]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:48.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T20:58:48.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: pgmap v96: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:47 smithi142 ceph-mon[34846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T20:58:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T20:58:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: pgmap v96: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:48.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T20:58:48.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T20:58:48.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T20:58:48.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T20:58:48.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:47 smithi050 ceph-mon[29546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T20:58:49.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:48 smithi050 ceph-mon[29546]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T20:58:49.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:48 smithi050 ceph-mon[29546]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T20:58:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 -- 172.21.15.50:0/3545641446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb418109ff0 msgr2=0x7fb41810c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 --2- 172.21.15.50:0/3545641446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb418109ff0 0x7fb41810c500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb41400b0a0 tx=0x7fb41400b3b0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 -- 172.21.15.50:0/3545641446 shutdown_connections 2024-05-10T20:58:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 --2- 172.21.15.50:0/3545641446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41806c2d0 0x7fb418106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 --2- 172.21.15.50:0/3545641446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb418109ff0 0x7fb41810c500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 --2- 172.21.15.50:0/3545641446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418107480 0x7fb418109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 -- 172.21.15.50:0/3545641446 >> 172.21.15.50:0/3545641446 conn(0x7fb4180fe880 msgr2=0x7fb418100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:58:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 -- 172.21.15.50:0/3545641446 shutdown_connections 2024-05-10T20:58:49.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.312+0000 7fb420800700 1 -- 172.21.15.50:0/3545641446 wait complete. 2024-05-10T20:58:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.313+0000 7fb420800700 1 Processor -- start 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 -- start start 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb41806c2d0 0x7fb41819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 0x7fb4181a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810fdd0 con 0x7fb41806c2d0 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810fc50 con 0x7fb418107480 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb420800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810ff50 con 0x7fb418109ff0 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb41d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb41de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb41806c2d0 0x7fb41819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb41d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40926/0 (socket says 172.21.15.50:40926) 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb41d400700 1 -- 172.21.15.50:0/3394672688 learned_addr learned my addr 172.21.15.50:0/3394672688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.314+0000 7fb41e800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 0x7fb4181a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 -- 172.21.15.50:0/3394672688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 msgr2=0x7fb4181a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 0x7fb4181a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 -- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb41806c2d0 msgr2=0x7fb41819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb41806c2d0 0x7fb41819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 -- 172.21.15.50:0/3394672688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181a45e0 con 0x7fb418107480 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41e800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 0x7fb4181a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb41d400700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9a0 tx=0x7fb40800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800cbd0 con 0x7fb418107480 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40800cd70 con 0x7fb418107480 2024-05-10T20:58:49.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408018680 con 0x7fb418107480 2024-05-10T20:58:49.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.315+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181a48d0 con 0x7fb418107480 2024-05-10T20:58:49.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.316+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a4dc0 con 0x7fb418107480 2024-05-10T20:58:49.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.317+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4180512c0 con 0x7fb418107480 2024-05-10T20:58:49.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.319+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb408010430 con 0x7fb418107480 2024-05-10T20:58:49.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.320+0000 7fb412a00700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 0x7fb4000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:58:49.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.321+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fb4080e10f0 con 0x7fb418107480 2024-05-10T20:58:49.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.321+0000 7fb41de00700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 0x7fb4000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:58:49.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.321+0000 7fb41de00700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 0x7fb4000b4e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009680 tx=0x7fb40c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:58:49.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.325+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4080aa140 con 0x7fb418107480 2024-05-10T20:58:49.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:48 smithi092 ceph-mon[34415]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T20:58:49.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:48 smithi092 ceph-mon[34415]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T20:58:49.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:48 smithi142 ceph-mon[34846]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T20:58:49.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:48 smithi142 ceph-mon[34846]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T20:58:49.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:58:49.571+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb41810be40 con 0x7fb4000b2900 2024-05-10T20:58:50.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:49 smithi092 ceph-mon[34415]: pgmap v98: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:50.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:49 smithi092 ceph-mon[34415]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:50.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:50.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:50.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:50.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:49 smithi142 ceph-mon[34846]: pgmap v98: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:50.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:49 smithi142 ceph-mon[34846]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:50.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:50.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:50.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:50.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:49 smithi050 ceph-mon[29546]: pgmap v98: 1 pgs: 1 active+clean; 320 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:50.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:49 smithi050 ceph-mon[29546]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:58:50.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:58:50.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:58:50.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:58:52.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:52 smithi050 ceph-mon[29546]: pgmap v99: 1 pgs: 1 active+clean; 320 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:52.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:52 smithi092 ceph-mon[34415]: pgmap v99: 1 pgs: 1 active+clean; 320 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:52.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:52 smithi142 ceph-mon[34846]: pgmap v99: 1 pgs: 1 active+clean; 320 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:58:54.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.8 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-05-10T20:58:54.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1739957638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]': finished 2024-05-10T20:58:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T20:58:54.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.8 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1739957638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]': finished 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T20:58:54.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.8 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1739957638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]: dispatch 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3deb55-a3e3-4b6d-9a12-6a411e9004ff"}]': finished 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T20:58:54.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:58:55.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:55 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2707483775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:55.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:55 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2707483775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:55.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:55 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2707483775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:58:56.860 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:56 smithi050 ceph-mon[29546]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-05-10T20:58:56.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:56 smithi092 ceph-mon[34415]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-05-10T20:58:56.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:56 smithi142 ceph-mon[34846]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-05-10T20:58:58.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:58:58 smithi092 ceph-mon[34415]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-05-10T20:58:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:58:58 smithi142 ceph-mon[34846]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-05-10T20:58:58.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:58:58 smithi050 ceph-mon[29546]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-05-10T20:59:00.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:00 smithi050 ceph-mon[29546]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:00.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:00 smithi092 ceph-mon[34415]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:00.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:00 smithi142 ceph-mon[34846]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:02 smithi092 ceph-mon[34415]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:02.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:02 smithi142 ceph-mon[34846]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:02.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:02 smithi050 ceph-mon[29546]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-05-10T20:59:03.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T20:59:03.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:03.879 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T20:59:03.879 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T20:59:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:04.750 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:04 smithi050 ceph-mon[29546]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:04.751 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:04 smithi050 ceph-mon[29546]: Deploying daemon osd.3 on smithi050 2024-05-10T20:59:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:04 smithi092 ceph-mon[34415]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:04 smithi092 ceph-mon[34415]: Deploying daemon osd.3 on smithi050 2024-05-10T20:59:04.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:04 smithi142 ceph-mon[34846]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:04.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:04 smithi142 ceph-mon[34846]: Deploying daemon osd.3 on smithi050 2024-05-10T20:59:06.867 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:06 smithi050 ceph-mon[29546]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:06.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:06 smithi092 ceph-mon[34415]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:06.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:06 smithi142 ceph-mon[34846]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:08.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:08 smithi050 ceph-mon[29546]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:08.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:08 smithi092 ceph-mon[34415]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:08 smithi142 ceph-mon[34846]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:10.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:10 smithi092 ceph-mon[34415]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:10.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:10 smithi142 ceph-mon[34846]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:10.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:10 smithi050 ceph-mon[29546]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:12.845 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:12 smithi050 ceph-mon[29546]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:12.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:12 smithi092 ceph-mon[34415]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:12.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:12 smithi142 ceph-mon[34846]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:14.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:14 smithi092 ceph-mon[34415]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:14.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:14.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:14 smithi142 ceph-mon[34846]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:14.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:14.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:14.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:14 smithi050 ceph-mon[29546]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:16.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:16 smithi092 ceph-mon[34415]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:16.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:16 smithi142 ceph-mon[34846]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:16.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:16 smithi050 ceph-mon[29546]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:18.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:17 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:17 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:17 smithi092 ceph-mon[34415]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:17 smithi092 ceph-mon[34415]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:17 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:17 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:17 smithi142 ceph-mon[34846]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:17 smithi142 ceph-mon[34846]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:17 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:17 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:17 smithi050 ceph-mon[29546]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:17 smithi050 ceph-mon[29546]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T20:59:18.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.683+0000 7fb412a00700 1 -- 172.21.15.50:0/3394672688 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb41810be40 con 0x7fb4000b2900 2024-05-10T20:59:18.690 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 3 on host 'smithi050' 2024-05-10T20:59:18.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.688+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 msgr2=0x7fb4000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:18.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.688+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 0x7fb4000b4e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009680 tx=0x7fb40c009260 comp rx=0 tx=0).stop 2024-05-10T20:59:18.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.688+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 msgr2=0x7fb41819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:18.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.688+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9a0 tx=0x7fb40800ecb0 comp rx=0 tx=0).stop 2024-05-10T20:59:18.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 shutdown_connections 2024-05-10T20:59:18.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4000b2900 0x7fb4000b4e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb418109ff0 0x7fb4181a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb41806c2d0 0x7fb41819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 --2- 172.21.15.50:0/3394672688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb418107480 0x7fb41819cfc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 >> 172.21.15.50:0/3394672688 conn(0x7fb4180fe880 msgr2=0x7fb418100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 shutdown_connections 2024-05-10T20:59:18.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T20:59:18.689+0000 7fb420800700 1 -- 172.21.15.50:0/3394672688 wait complete. 2024-05-10T20:59:18.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:18.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:18.828 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:19.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T20:59:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T20:59:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:19.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:19.475 DEBUG:teuthology.orchestra.run.smithi050:osd.3> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T20:59:19.479 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-05-10T20:59:19.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T20:59:19.719 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T20:59:20.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:19 smithi050 ceph-mon[29546]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:59:20.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:19 smithi050 ceph-mon[29546]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T20:59:20.050 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:20.050 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:20.087 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:19 smithi092 ceph-mon[34415]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:59:20.087 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:19 smithi092 ceph-mon[34415]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T20:59:20.087 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:20.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:20.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:19 smithi142 ceph-mon[34846]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T20:59:20.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:19 smithi142 ceph-mon[34846]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T20:59:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:21.023 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T20:59:21.023 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T20:59:21.023 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:21.024 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] boot 2024-05-10T20:59:21.024 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T20:59:21.024 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] boot 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T20:59:21.064 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:21.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T20:59:21.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T20:59:21.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-10T20:59:21.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: osd.3 [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] boot 2024-05-10T20:59:21.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: osdmap e25: 4 total, 4 up, 4 in 2024-05-10T20:59:21.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133341 s, 786 MB/s 2024-05-10T20:59:21.655 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-10T20:59:21.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:21 smithi092 ceph-mon[34415]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T20:59:21.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:21 smithi092 ceph-mon[34415]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:21.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:21 smithi092 ceph-mon[34415]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T20:59:22.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:21 smithi142 ceph-mon[34846]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T20:59:22.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:21 smithi142 ceph-mon[34846]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:22.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:21 smithi142 ceph-mon[34846]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T20:59:22.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:21 smithi050 ceph-mon[29546]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T20:59:22.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:21 smithi050 ceph-mon[29546]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:22.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:21 smithi050 ceph-mon[29546]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T20:59:22.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-05-10T20:59:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 -- 172.21.15.92:0/461480785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 msgr2=0x7f967010b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 --2- 172.21.15.92:0/461480785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f967010b5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f966800b0a0 tx=0x7f966800b3b0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 -- 172.21.15.92:0/461480785 shutdown_connections 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 --2- 172.21.15.92:0/461480785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f967010b5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 --2- 172.21.15.92:0/461480785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 0x7f9670103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 --2- 172.21.15.92:0/461480785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 0x7f9670104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 -- 172.21.15.92:0/461480785 >> 172.21.15.92:0/461480785 conn(0x7f96700fe880 msgr2=0x7f9670100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:59:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 -- 172.21.15.92:0/461480785 shutdown_connections 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.025+0000 7f9675363700 1 -- 172.21.15.92:0/461480785 wait complete. 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.026+0000 7f9675363700 1 Processor -- start 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.026+0000 7f9675363700 1 -- start start 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 0x7f9670199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 0x7f9670199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f967010ed20 con 0x7f9670103240 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f967010eba0 con 0x7f9670104d10 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f967010eea0 con 0x7f9670104370 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42298/0 (socket says 172.21.15.92:42298) 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 -- 172.21.15.92:0/3783995391 learned_addr learned my addr 172.21.15.92:0/3783995391 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966e7fc700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 0x7f9670199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:24.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966effd700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 0x7f9670199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 -- 172.21.15.92:0/3783995391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 msgr2=0x7f9670199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 0x7f9670199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 -- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 msgr2=0x7f9670199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 0x7f9670199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 -- 172.21.15.92:0/3783995391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96701a1370 con 0x7f9670104d10 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f966f7fe700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f966800b070 tx=0x7f9668004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668021070 con 0x7f9670104d10 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f966801d900 con 0x7f9670104d10 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96701a1600 con 0x7f9670104d10 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.027+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668005390 con 0x7f9670104d10 2024-05-10T20:59:24.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.028+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96701a1bc0 con 0x7f9670104d10 2024-05-10T20:59:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.029+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96700512c0 con 0x7f9670104d10 2024-05-10T20:59:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.030+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9668024070 con 0x7f9670104d10 2024-05-10T20:59:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.031+0000 7f9667fff700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 0x7f965c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.031+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f966800f020 con 0x7f9670104d10 2024-05-10T20:59:24.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.031+0000 7f966effd700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 0x7f965c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:24.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.032+0000 7f966effd700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 0x7f965c0b4d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f9660005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:59:24.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.035+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f966802b020 con 0x7f9670104d10 2024-05-10T20:59:24.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:24.248+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9670063c00 con 0x7f965c0b2880 2024-05-10T20:59:24.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:24 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:24 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:24.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:24 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:25.857 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:25 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi050 2024-05-10T20:59:25.857 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:25 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi050 to 8547M 2024-05-10T20:59:25.857 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:25 smithi092 ceph-mon[34415]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:25.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:25 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi050 2024-05-10T20:59:25.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:25 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi050 to 8547M 2024-05-10T20:59:25.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:25 smithi142 ceph-mon[34846]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:25 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi050 2024-05-10T20:59:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:25 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi050 to 8547M 2024-05-10T20:59:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:25 smithi050 ceph-mon[29546]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:26.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:26 smithi092 ceph-mon[34415]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T20:59:26.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:26 smithi142 ceph-mon[34846]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T20:59:26.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:26 smithi050 ceph-mon[29546]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T20:59:27.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:27 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/316315108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:27 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:27 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]': finished 2024-05-10T20:59:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:27 smithi092 ceph-mon[34415]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T20:59:27.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:27 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:27 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/316315108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:27 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:27 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]': finished 2024-05-10T20:59:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:27 smithi142 ceph-mon[34846]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T20:59:27.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:27 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:27 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/316315108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:27 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]: dispatch 2024-05-10T20:59:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:27 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "352735a0-8b3b-47b4-accb-d76af31a6279"}]': finished 2024-05-10T20:59:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:27 smithi050 ceph-mon[29546]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T20:59:27.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:27 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:28 smithi092 ceph-mon[34415]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T20:59:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:28 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/2690743183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:28 smithi142 ceph-mon[34846]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T20:59:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:28 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/2690743183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:28 smithi050 ceph-mon[29546]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T20:59:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:28 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/2690743183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:30.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:30 smithi092 ceph-mon[34415]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T20:59:30.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:30 smithi142 ceph-mon[34846]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T20:59:30.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:30 smithi050 ceph-mon[29546]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T20:59:32.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:32 smithi092 ceph-mon[34415]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T20:59:32.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:32 smithi142 ceph-mon[34846]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T20:59:32.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:32 smithi050 ceph-mon[29546]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T20:59:34.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:34 smithi092 ceph-mon[34415]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:34.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T20:59:34.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:34.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:34 smithi142 ceph-mon[34846]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:34.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T20:59:34.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:34 smithi050 ceph-mon[29546]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T20:59:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:35.624 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:35 smithi092 ceph-mon[34415]: Deploying daemon osd.4 on smithi092 2024-05-10T20:59:35.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:35 smithi142 ceph-mon[34846]: Deploying daemon osd.4 on smithi092 2024-05-10T20:59:35.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:35 smithi050 ceph-mon[29546]: Deploying daemon osd.4 on smithi092 2024-05-10T20:59:36.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:36 smithi092 ceph-mon[34415]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:36.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:36 smithi142 ceph-mon[34846]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:36.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:36 smithi050 ceph-mon[29546]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:38.846 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:38 smithi092 ceph-mon[34415]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:38.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:38 smithi142 ceph-mon[34846]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:38.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:38 smithi050 ceph-mon[29546]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:40.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:40 smithi092 ceph-mon[34415]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:40.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:40.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:40 smithi142 ceph-mon[34846]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:40.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:40 smithi050 ceph-mon[29546]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:42 smithi092 ceph-mon[34415]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:42.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:42 smithi142 ceph-mon[34846]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:42.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:42 smithi050 ceph-mon[29546]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:43.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.636+0000 7f9667fff700 1 -- 172.21.15.92:0/3783995391 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9670063c00 con 0x7f965c0b2880 2024-05-10T20:59:43.643 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 msgr2=0x7f965c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 0x7f965c0b4d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f9660005dd0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 msgr2=0x7f96701a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f966800b070 tx=0x7f9668004ef0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 shutdown_connections 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f965c0b2880 0x7f965c0b4d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9670104d10 0x7f96701a0cf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9670103240 0x7f9670199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 --2- 172.21.15.92:0/3783995391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9670104370 0x7f9670199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 >> 172.21.15.92:0/3783995391 conn(0x7f96700fe880 msgr2=0x7f96701075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:59:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 shutdown_connections 2024-05-10T20:59:43.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T20:59:43.639+0000 7f9675363700 1 -- 172.21.15.92:0/3783995391 wait complete. 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:44.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:44 smithi092 ceph-mon[34415]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.257 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T20:59:44.259 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_3... 2024-05-10T20:59:44.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:44 smithi142 ceph-mon[34846]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:44 smithi050 ceph-mon[29546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T20:59:44.501 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:59:45.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:45 smithi142 ceph-mon[34846]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T20:59:45.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:45 smithi142 ceph-mon[34846]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:45.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:45 smithi142 ceph-mon[34846]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T20:59:45.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:45.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:45 smithi142 ceph-mon[34846]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:45.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:45 smithi092 ceph-mon[34415]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T20:59:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:45 smithi092 ceph-mon[34415]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:45 smithi092 ceph-mon[34415]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T20:59:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:45 smithi092 ceph-mon[34415]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:45.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:45 smithi050 ceph-mon[29546]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T20:59:45.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:45 smithi050 ceph-mon[29546]: from='osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:45.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:45 smithi050 ceph-mon[29546]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T20:59:45.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:45.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:45 smithi050 ceph-mon[29546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T20:59:46.354 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T20:59:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T20:59:46.355 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T20:59:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T20:59:46.355 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116025 s, 904 MB/s 2024-05-10T20:59:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T20:59:46.733 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:46.733 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:59:46.733 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T20:59:46.733 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:46.733 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: from='osd.4 ' entity='osd.4' 2024-05-10T20:59:46.734 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: from='osd.4 ' entity='osd.4' 2024-05-10T20:59:46.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:46.841 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 20:59:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T20:59:46.502+0000 7f22e378b700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T20:59:46.841 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 20:59:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T20:59:46.634+0000 7f22db538700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T20:59:46.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:46.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T20:59:46.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T20:59:46.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:46.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: from='osd.4 ' entity='osd.4' 2024-05-10T20:59:46.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:47.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-05-10T20:59:47.303 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.887 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T20:59:48.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 -- 172.21.15.142:0/4234979111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103430 msgr2=0x7fdaac103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 --2- 172.21.15.142:0/4234979111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103430 0x7fdaac103840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009990 tx=0x7fda9c009ca0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 -- 172.21.15.142:0/4234979111 shutdown_connections 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 --2- 172.21.15.142:0/4234979111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103d80 0x7fdaac10b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 --2- 172.21.15.142:0/4234979111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaac104720 0x7fdaac104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 --2- 172.21.15.142:0/4234979111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103430 0x7fdaac103840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 -- 172.21.15.142:0/4234979111 >> 172.21.15.142:0/4234979111 conn(0x7fdaac0fe010 msgr2=0x7fdaac1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 -- 172.21.15.142:0/4234979111 shutdown_connections 2024-05-10T20:59:48.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.745+0000 7fdab446f700 1 -- 172.21.15.142:0/4234979111 wait complete. 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 Processor -- start 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 -- start start 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103430 0x7fdaac19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaac104720 0x7fdaac1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10ed30 con 0x7fdaac104720 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10ebb0 con 0x7fdaac103430 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab446f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10eeb0 con 0x7fdaac103d80 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab1a0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab1a0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:60286/0 (socket says 172.21.15.142:60286) 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab1a0a700 1 -- 172.21.15.142:0/2984088354 learned_addr learned my addr 172.21.15.142:0/2984088354 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T20:59:48.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.746+0000 7fdab220b700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103430 0x7fdaac19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 -- 172.21.15.142:0/2984088354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103430 msgr2=0x7fdaac19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103430 0x7fdaac19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 -- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaac104720 msgr2=0x7fdaac1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaac104720 0x7fdaac1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 -- 172.21.15.142:0/2984088354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac1a2520 con 0x7fdaac103d80 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab1a0a700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c027070 con 0x7fdaac103d80 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda9c0040c0 con 0x7fdaac103d80 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a27b0 con 0x7fdaac103d80 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c022d90 con 0x7fdaac103d80 2024-05-10T20:59:48.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.747+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a2c80 con 0x7fdaac103d80 2024-05-10T20:59:48.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.748+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac0512c0 con 0x7fdaac103d80 2024-05-10T20:59:48.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.750+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fda9c02a070 con 0x7fdaac103d80 2024-05-10T20:59:48.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.751+0000 7fdaa37fe700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T20:59:48.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.751+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fda9c0e2300 con 0x7fdaac103d80 2024-05-10T20:59:48.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.751+0000 7fdab220b700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T20:59:48.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.752+0000 7fdab220b700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 0x7fda980b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdaac06b430 tx=0x7fdaa800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T20:59:48.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.755+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda9c013070 con 0x7fdaac103d80 2024-05-10T20:59:48.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi092 2024-05-10T20:59:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-10T20:59:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] boot 2024-05-10T20:59:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T20:59:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:48.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T20:59:48.966+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdaac1a2fa0 con 0x7fda980b2880 2024-05-10T20:59:49.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi092 2024-05-10T20:59:49.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-10T20:59:49.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:49.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] boot 2024-05-10T20:59:49.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T20:59:49.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi092 2024-05-10T20:59:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-10T20:59:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-10T20:59:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: osd.4 [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] boot 2024-05-10T20:59:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: osdmap e31: 5 total, 5 up, 5 in 2024-05-10T20:59:49.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T20:59:49.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:49 smithi142 ceph-mon[34846]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T20:59:49.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:49.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:49.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:50.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:49 smithi092 ceph-mon[34415]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T20:59:50.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:50.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:50.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:50.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:49 smithi050 ceph-mon[29546]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T20:59:50.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T20:59:50.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T20:59:50.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T20:59:50.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:50 smithi142 ceph-mon[34846]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:50.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:50 smithi142 ceph-mon[34846]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:51.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:50 smithi092 ceph-mon[34415]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:51.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:50 smithi092 ceph-mon[34415]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:50 smithi050 ceph-mon[29546]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T20:59:51.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:50 smithi050 ceph-mon[29546]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:52.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:51 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/726290769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:51 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:51 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]': finished 2024-05-10T20:59:52.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:51 smithi092 ceph-mon[34415]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T20:59:52.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:51 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T20:59:52.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:51 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/726290769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:51 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:51 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]': finished 2024-05-10T20:59:52.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:51 smithi142 ceph-mon[34846]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T20:59:52.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:51 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T20:59:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:51 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/726290769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:51 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]: dispatch 2024-05-10T20:59:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:51 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23efb5de-d1d9-4654-ab57-81f4a6187db5"}]': finished 2024-05-10T20:59:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:51 smithi050 ceph-mon[29546]: osdmap e33: 6 total, 5 up, 6 in 2024-05-10T20:59:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:51 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T20:59:53.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:52 smithi092 ceph-mon[34415]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:53.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:52 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/2143188172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:52 smithi142 ceph-mon[34846]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:52 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/2143188172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:53.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:52 smithi050 ceph-mon[29546]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:53.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:52 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/2143188172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T20:59:55.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:54 smithi092 ceph-mon[34415]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:55.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:54 smithi142 ceph-mon[34846]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:55.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:54 smithi050 ceph-mon[29546]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:57.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:56 smithi092 ceph-mon[34415]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:57.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:56 smithi142 ceph-mon[34846]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:57.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:56 smithi050 ceph-mon[29546]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:58 smithi092 ceph-mon[34415]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:58 smithi142 ceph-mon[34846]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T20:59:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:58 smithi050 ceph-mon[29546]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:00.025 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:59 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T21:00:00.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:59 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:00.026 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 20:59:59 smithi142 ceph-mon[34846]: Deploying daemon osd.5 on smithi142 2024-05-10T21:00:00.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:59 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T21:00:00.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:59 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:00.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 20:59:59 smithi092 ceph-mon[34415]: Deploying daemon osd.5 on smithi142 2024-05-10T21:00:00.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T21:00:00.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:00.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 20:59:59 smithi050 ceph-mon[29546]: Deploying daemon osd.5 on smithi142 2024-05-10T21:00:00.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:00 smithi142 ceph-mon[34846]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:00.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:00 smithi142 ceph-mon[34846]: overall HEALTH_OK 2024-05-10T21:00:01.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:00 smithi092 ceph-mon[34415]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:01.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:00 smithi092 ceph-mon[34415]: overall HEALTH_OK 2024-05-10T21:00:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:00 smithi050 ceph-mon[29546]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:00 smithi050 ceph-mon[29546]: overall HEALTH_OK 2024-05-10T21:00:03.020 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:02 smithi142 ceph-mon[34846]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:03.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:02 smithi092 ceph-mon[34415]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:03.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:02 smithi050 ceph-mon[29546]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:04.803 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:04 smithi142 ceph-mon[34846]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:04.803 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:04.803 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:04.803 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:05.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:04 smithi092 ceph-mon[34415]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:05.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:05.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:05.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:05.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:04 smithi050 ceph-mon[29546]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:05.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:05.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:05.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:07.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:06 smithi092 ceph-mon[34415]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:07.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:06 smithi142 ceph-mon[34846]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:06 smithi050 ceph-mon[29546]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:07.606 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-05-10T21:00:07.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.600+0000 7fdaa37fe700 1 -- 172.21.15.142:0/2984088354 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaac1a2fa0 con 0x7fda980b2880 2024-05-10T21:00:07.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 msgr2=0x7fda980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:07.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 0x7fda980b4d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdaac06b430 tx=0x7fdaa800d040 comp rx=0 tx=0).stop 2024-05-10T21:00:07.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 msgr2=0x7fdaac19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:07.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0047b0 comp rx=0 tx=0).stop 2024-05-10T21:00:07.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 shutdown_connections 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fda980b2880 0x7fda980b4d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaac103430 0x7fdaac19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaac104720 0x7fdaac1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 --2- 172.21.15.142:0/2984088354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaac103d80 0x7fdaac19af00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 >> 172.21.15.142:0/2984088354 conn(0x7fdaac0fe010 msgr2=0x7fdaac0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 shutdown_connections 2024-05-10T21:00:07.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:00:07.602+0000 7fdab446f700 1 -- 172.21.15.142:0/2984088354 wait complete. 2024-05-10T21:00:08.206 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T21:00:08.208 INFO:tasks.cephadm:Deploying osd.6 on smithi050 with /dev/vg_nvme/lv_2... 2024-05-10T21:00:08.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T21:00:08.679 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:08.839 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:08 smithi050 ceph-mon[29546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:08.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:08.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:08 smithi092 ceph-mon[34415]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:08.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:08.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:08 smithi142 ceph-mon[34846]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T21:00:09.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:09 smithi142 ceph-mon[34846]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T21:00:09.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:09 smithi142 ceph-mon[34846]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T21:00:09.796 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:09.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:09 smithi142 ceph-mon[34846]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:09.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:09 smithi142 ceph-mon[34846]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:09.797 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:00:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:00:09.749+0000 7f27ae00b700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T21:00:09.797 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:00:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:00:09.757+0000 7f27a9dc0700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T21:00:09.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:09 smithi050 ceph-mon[29546]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T21:00:09.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:09 smithi050 ceph-mon[29546]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T21:00:09.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:09.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:09 smithi050 ceph-mon[29546]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:09.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:09 smithi050 ceph-mon[29546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:10.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:09 smithi092 ceph-mon[34415]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T21:00:10.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:09 smithi092 ceph-mon[34415]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T21:00:10.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:10.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:09 smithi092 ceph-mon[34415]: from='osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:10.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:09 smithi092 ceph-mon[34415]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:00:10.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:10 smithi142 ceph-mon[34846]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:10.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:10 smithi142 ceph-mon[34846]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:00:10.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:10 smithi142 ceph-mon[34846]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T21:00:10.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:10.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:10 smithi050 ceph-mon[29546]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:10.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:10 smithi050 ceph-mon[29546]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:00:10.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:10 smithi050 ceph-mon[29546]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T21:00:10.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:10 smithi092 ceph-mon[34415]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-10T21:00:11.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:10 smithi092 ceph-mon[34415]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:00:11.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:10 smithi092 ceph-mon[34415]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T21:00:11.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:11.588 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T21:00:11.589 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T21:00:11.589 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-10T21:00:11.589 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-10T21:00:11.589 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.037404 s, 280 MB/s 2024-05-10T21:00:11.589 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-10T21:00:12.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:00:12.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:00:12.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] boot 2024-05-10T21:00:12.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T21:00:12.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:11 smithi050 ceph-mon[29546]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] boot 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:11 smithi092 ceph-mon[34415]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T21:00:12.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: osd.5 [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] boot 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: osdmap e36: 6 total, 6 up, 6 in 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:12.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:12.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:12.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:12.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:11 smithi142 ceph-mon[34846]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T21:00:12.520 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-05-10T21:00:12.962 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:00:13.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:12 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi142 2024-05-10T21:00:13.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:12 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-10T21:00:13.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:12 smithi050 ceph-mon[29546]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:13.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:12 smithi050 ceph-mon[29546]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T21:00:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:12 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi142 2024-05-10T21:00:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:12 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-10T21:00:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:12 smithi092 ceph-mon[34415]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:12 smithi092 ceph-mon[34415]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T21:00:13.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:12 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi142 2024-05-10T21:00:13.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:12 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-10T21:00:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:12 smithi142 ceph-mon[34846]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:12 smithi142 ceph-mon[34846]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T21:00:14.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:14 smithi050 ceph-mon[29546]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:15.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:14 smithi092 ceph-mon[34415]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:15.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:14 smithi142 ceph-mon[34846]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 -- 172.21.15.50:0/3950417096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10c420 msgr2=0x7f4cdc112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/3950417096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10c420 0x7f4cdc112cd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400b0a0 tx=0x7f4cd400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 -- 172.21.15.50:0/3950417096 shutdown_connections 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/3950417096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10c420 0x7f4cdc112cd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/3950417096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10ab10 0x7f4cdc10af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/3950417096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 -- 172.21.15.50:0/3950417096 >> 172.21.15.50:0/3950417096 conn(0x7f4cdc0799c0 msgr2=0x7f4cdc07be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 -- 172.21.15.50:0/3950417096 shutdown_connections 2024-05-10T21:00:15.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.210+0000 7f4ce1a00700 1 -- 172.21.15.50:0/3950417096 wait complete. 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.211+0000 7f4ce1a00700 1 Processor -- start 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.211+0000 7f4ce1a00700 1 -- start start 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10ab10 0x7f4cdc1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10c420 0x7f4cdc1a8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1163c0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc116240 con 0x7f4cdc10ab10 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4ce1a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc116540 con 0x7f4cdc10c420 2024-05-10T21:00:15.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4cda000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4cda000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56374/0 (socket says 172.21.15.50:56374) 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4cda000700 1 -- 172.21.15.50:0/1300142918 learned_addr learned my addr 172.21.15.50:0/1300142918 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4cdaa00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10ab10 0x7f4cdc1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.212+0000 7f4cdb400700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10c420 0x7f4cdc1a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 -- 172.21.15.50:0/1300142918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10c420 msgr2=0x7f4cdc1a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10c420 0x7f4cdc1a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 -- 172.21.15.50:0/1300142918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10ab10 msgr2=0x7f4cdc1a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10ab10 0x7f4cdc1a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 -- 172.21.15.50:0/1300142918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc1a89d0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cda000700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9a0 tx=0x7f4cc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400ca10 con 0x7f4cdc10ba80 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.213+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc400cbb0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.214+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4018680 con 0x7f4cdc10ba80 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.214+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc1a8cc0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.214+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1a91b0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.215+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc0512c0 con 0x7f4cdc10ba80 2024-05-10T21:00:15.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.218+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4cc4010430 con 0x7f4cdc10ba80 2024-05-10T21:00:15.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.219+0000 7f4cd3400700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 0x7f4cc00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:15.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.220+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f4cc40e1840 con 0x7f4cdc10ba80 2024-05-10T21:00:15.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.220+0000 7f4cdaa00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 0x7f4cc00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:15.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.221+0000 7f4cdaa00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 0x7f4cc00b4e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc10b9b0 tx=0x7f4ccc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:00:15.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.223+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cc40aa380 con 0x7f4cdc10ba80 2024-05-10T21:00:15.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:15.471+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4cdc063c00 con 0x7f4cc00b2900 2024-05-10T21:00:16.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:16.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:16.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:16.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:16.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:17.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:16 smithi092 ceph-mon[34415]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:17.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:16 smithi092 ceph-mon[34415]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:17.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:16 smithi142 ceph-mon[34846]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:17.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:16 smithi142 ceph-mon[34846]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:17.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:16 smithi050 ceph-mon[29546]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:17.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:16 smithi050 ceph-mon[29546]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:19.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:18 smithi092 ceph-mon[34415]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-10T21:00:19.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:18 smithi142 ceph-mon[34846]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-10T21:00:19.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:18 smithi050 ceph-mon[29546]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-10T21:00:20.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4230851107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:19 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:19 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]': finished 2024-05-10T21:00:20.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:19 smithi092 ceph-mon[34415]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T21:00:20.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4230851107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:19 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:19 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]': finished 2024-05-10T21:00:20.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:19 smithi142 ceph-mon[34846]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T21:00:20.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:20.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4230851107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:19 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]: dispatch 2024-05-10T21:00:20.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:19 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1"}]': finished 2024-05-10T21:00:20.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:19 smithi050 ceph-mon[29546]: osdmap e39: 7 total, 6 up, 7 in 2024-05-10T21:00:20.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:21.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:20 smithi092 ceph-mon[34415]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:00:21.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1759003580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:21.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:20 smithi142 ceph-mon[34846]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:00:21.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1759003580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:21.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:20 smithi050 ceph-mon[29546]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:00:21.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1759003580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:23.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:22 smithi092 ceph-mon[34415]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:00:23.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:22 smithi142 ceph-mon[34846]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:00:23.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:22 smithi050 ceph-mon[29546]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:00:25.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:24 smithi050 ceph-mon[29546]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:25.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:24 smithi092 ceph-mon[34415]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:25.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:24 smithi142 ceph-mon[34846]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:27.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:26 smithi092 ceph-mon[34415]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:27.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:26 smithi142 ceph-mon[34846]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:27.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:26 smithi050 ceph-mon[29546]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:00:29.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:28 smithi092 ceph-mon[34415]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:29.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:28 smithi142 ceph-mon[34846]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:29.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:28 smithi050 ceph-mon[29546]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:30.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:29 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T21:00:30.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:29 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:30.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:29 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T21:00:30.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:29 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:29 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T21:00:30.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:29 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:31.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:30 smithi092 ceph-mon[34415]: Deploying daemon osd.6 on smithi050 2024-05-10T21:00:31.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:30 smithi092 ceph-mon[34415]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:31.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:30 smithi142 ceph-mon[34846]: Deploying daemon osd.6 on smithi050 2024-05-10T21:00:31.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:30 smithi142 ceph-mon[34846]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:31.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:30 smithi050 ceph-mon[29546]: Deploying daemon osd.6 on smithi050 2024-05-10T21:00:31.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:30 smithi050 ceph-mon[29546]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:32.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:31 smithi092 ceph-mon[34415]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:32.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:31 smithi142 ceph-mon[34846]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:32.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:31 smithi050 ceph-mon[29546]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:34.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:34 smithi092 ceph-mon[34415]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:34.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:34 smithi142 ceph-mon[34846]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:34 smithi050 ceph-mon[29546]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:36.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:36 smithi092 ceph-mon[34415]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:36.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:36 smithi142 ceph-mon[34846]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:36.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:36 smithi050 ceph-mon[29546]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:38.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:38 smithi050 ceph-mon[29546]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:38.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:38 smithi092 ceph-mon[34415]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:38.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:38 smithi142 ceph-mon[34846]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:40.222 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.222 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.222 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:40.222 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:39 smithi050 ceph-mon[29546]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:40.229 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.229 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.229 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:40.229 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:39 smithi142 ceph-mon[34846]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:40.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:40.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:40.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:39 smithi092 ceph-mon[34415]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:42.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:42 smithi050 ceph-mon[29546]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:42 smithi092 ceph-mon[34415]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:42.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:42 smithi142 ceph-mon[34846]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:43 smithi050 ceph-mon[29546]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:43.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:43 smithi050 ceph-mon[29546]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:43 smithi092 ceph-mon[34415]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:43 smithi092 ceph-mon[34415]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:44.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:44.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:44.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:43 smithi142 ceph-mon[34846]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:44.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:43 smithi142 ceph-mon[34846]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T21:00:44.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.877+0000 7f4cd3400700 1 -- 172.21.15.50:0/1300142918 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4cdc063c00 con 0x7f4cc00b2900 2024-05-10T21:00:44.885 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 6 on host 'smithi050' 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 msgr2=0x7f4cc00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 0x7f4cc00b4e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc10b9b0 tx=0x7f4ccc009260 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 msgr2=0x7f4cdc1a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9a0 tx=0x7f4cc400ecb0 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 shutdown_connections 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b2900 0x7f4cc00b4e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cdc10ab10 0x7f4cdc1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc10c420 0x7f4cdc1a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 --2- 172.21.15.50:0/1300142918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cdc10ba80 0x7f4cdc1a13b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 >> 172.21.15.50:0/1300142918 conn(0x7f4cdc0799c0 msgr2=0x7f4cdc10ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:00:44.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 shutdown_connections 2024-05-10T21:00:44.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:00:44.882+0000 7f4ce1a00700 1 -- 172.21.15.50:0/1300142918 wait complete. 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:45.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:45.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:45.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:00:45.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:45.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:45.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:45.752 DEBUG:teuthology.orchestra.run.smithi050:osd.6> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T21:00:45.757 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-05-10T21:00:45.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:45.984 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:46.010 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:00:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:00:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T21:00:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:00:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T21:00:46.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:46.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] boot 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T21:00:47.040 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] boot 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T21:00:47.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: osd.6 [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] boot 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: osdmap e42: 7 total, 7 up, 7 in 2024-05-10T21:00:47.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132592 s, 791 MB/s 2024-05-10T21:00:47.874 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-10T21:00:47.898 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:47 smithi092 ceph-mon[34415]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T21:00:48.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:47 smithi142 ceph-mon[34846]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T21:00:48.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:47 smithi050 ceph-mon[29546]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T21:00:48.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-05-10T21:00:48.775 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:00:49.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:48 smithi092 ceph-mon[34415]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:49.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:48 smithi142 ceph-mon[34846]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:49.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:48 smithi050 ceph-mon[29546]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:49.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:49 smithi050 ceph-mon[29546]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:50.055 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:49 smithi092 ceph-mon[34415]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:50.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:49 smithi142 ceph-mon[34846]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 -- 172.21.15.92:0/204668628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104d00 msgr2=0x7eff8c10b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 --2- 172.21.15.92:0/204668628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104d00 0x7eff8c10b5b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff84009990 tx=0x7eff84009ca0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 -- 172.21.15.92:0/204668628 shutdown_connections 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 --2- 172.21.15.92:0/204668628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104d00 0x7eff8c10b5b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 --2- 172.21.15.92:0/204668628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c103230 0x7eff8c103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 --2- 172.21.15.92:0/204668628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c104360 0x7eff8c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 -- 172.21.15.92:0/204668628 >> 172.21.15.92:0/204668628 conn(0x7eff8c0fe890 msgr2=0x7eff8c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 -- 172.21.15.92:0/204668628 shutdown_connections 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.255+0000 7eff92ffb700 1 -- 172.21.15.92:0/204668628 wait complete. 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 Processor -- start 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 -- start start 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 0x7eff8c076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c104d00 0x7eff8c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10ed10 con 0x7eff8c104d00 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10eb90 con 0x7eff8c104360 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff92ffb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10ee90 con 0x7eff8c103230 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34654/0 (socket says 172.21.15.92:34654) 2024-05-10T21:00:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 -- 172.21.15.92:0/3967583593 learned_addr learned my addr 172.21.15.92:0/3967583593 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff90d97700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 0x7eff8c076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 -- 172.21.15.92:0/3967583593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 msgr2=0x7eff8c076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 0x7eff8c076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 -- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c104d00 msgr2=0x7eff8c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c104d00 0x7eff8c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 -- 172.21.15.92:0/3967583593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c1a2480 con 0x7eff8c104360 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff90d97700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 0x7eff8c076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.256+0000 7eff83fff700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00d950 tx=0x7eff7c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.257+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c016070 con 0x7eff8c104360 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.257+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7c010460 con 0x7eff8c104360 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.257+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00f620 con 0x7eff8c104360 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.257+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1a2770 con 0x7eff8c104360 2024-05-10T21:00:50.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.257+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a2d80 con 0x7eff8c104360 2024-05-10T21:00:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.258+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c0512c0 con 0x7eff8c104360 2024-05-10T21:00:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.259+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff7c019070 con 0x7eff8c104360 2024-05-10T21:00:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.260+0000 7eff81ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 0x7eff740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:00:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.260+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7eff7c0e10b0 con 0x7eff8c104360 2024-05-10T21:00:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.260+0000 7eff90d97700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 0x7eff740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:00:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.261+0000 7eff90d97700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 0x7eff740b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff8c104260 tx=0x7eff88009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:00:50.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.264+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff7c0a9ae0 con 0x7eff8c104360 2024-05-10T21:00:50.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:00:50.474+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff8c063c00 con 0x7eff740b2900 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi050 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi050 to 5698M 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:51.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi050 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi050 to 5698M 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:51.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi050 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi050 to 5698M 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:00:51.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:00:52.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:51 smithi092 ceph-mon[34415]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:52.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:51 smithi142 ceph-mon[34846]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:52.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:51 smithi050 ceph-mon[29546]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:53.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:53 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/2776606148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:53 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:53 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]': finished 2024-05-10T21:00:53.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:53 smithi092 ceph-mon[34415]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T21:00:53.549 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:00:53.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:53 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/2776606148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:53 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:53 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]': finished 2024-05-10T21:00:53.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:53 smithi142 ceph-mon[34846]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T21:00:53.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:00:53.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:53 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/2776606148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:53 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]: dispatch 2024-05-10T21:00:53.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:53 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d87384-c4c0-4d95-b34e-bd0ff0680647"}]': finished 2024-05-10T21:00:53.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:53 smithi050 ceph-mon[29546]: osdmap e44: 8 total, 7 up, 8 in 2024-05-10T21:00:53.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:00:54.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:54 smithi092 ceph-mon[34415]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:54.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:54 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/1313041888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:54.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:54 smithi142 ceph-mon[34846]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:54.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:54 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/1313041888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:54.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:54 smithi050 ceph-mon[29546]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:54.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:54 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/1313041888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:00:56.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:56 smithi092 ceph-mon[34415]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:56.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:56 smithi142 ceph-mon[34846]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:56.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:56 smithi050 ceph-mon[29546]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:58.674 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:00:58 smithi092 ceph-mon[34415]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:00:58 smithi142 ceph-mon[34846]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:00:58.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:00:58 smithi050 ceph-mon[29546]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:00.651 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:00 smithi092 ceph-mon[34415]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:00.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:00 smithi142 ceph-mon[34846]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:00.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:00 smithi050 ceph-mon[29546]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:01.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T21:01:01.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:01.612 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:01 smithi092 ceph-mon[34415]: Deploying daemon osd.7 on smithi092 2024-05-10T21:01:01.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:01 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T21:01:01.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:01 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:01.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:01 smithi142 ceph-mon[34846]: Deploying daemon osd.7 on smithi092 2024-05-10T21:01:01.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T21:01:01.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:01.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:01 smithi050 ceph-mon[29546]: Deploying daemon osd.7 on smithi092 2024-05-10T21:01:02.848 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:02 smithi092 ceph-mon[34415]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:02.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:02 smithi142 ceph-mon[34846]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:02.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:02 smithi050 ceph-mon[29546]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:04.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:04 smithi092 ceph-mon[34415]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:04.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:04 smithi142 ceph-mon[34846]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:04.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:04 smithi050 ceph-mon[29546]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:06.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:06 smithi092 ceph-mon[34415]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:06.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:06.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:06 smithi142 ceph-mon[34846]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:06.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:06 smithi050 ceph-mon[29546]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:08.706 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:08 smithi092 ceph-mon[34415]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:08.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:08 smithi142 ceph-mon[34846]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:08.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:08 smithi050 ceph-mon[29546]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:09.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.876+0000 7eff81ffb700 1 -- 172.21.15.92:0/3967583593 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff8c063c00 con 0x7eff740b2900 2024-05-10T21:01:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-05-10T21:01:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 msgr2=0x7eff740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 0x7eff740b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff8c104260 tx=0x7eff88009570 comp rx=0 tx=0).stop 2024-05-10T21:01:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 msgr2=0x7eff8c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:09.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00d950 tx=0x7eff7c00dd10 comp rx=0 tx=0).stop 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 shutdown_connections 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7eff740b2900 0x7eff740b4e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c104d00 0x7eff8c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff8c103230 0x7eff8c076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.878+0000 7eff92ffb700 1 --2- 172.21.15.92:0/3967583593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c104360 0x7eff8c19f6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:09.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.879+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 >> 172.21.15.92:0/3967583593 conn(0x7eff8c0fe890 msgr2=0x7eff8c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:01:09.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.879+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 shutdown_connections 2024-05-10T21:01:09.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:01:09.879+0000 7eff92ffb700 1 -- 172.21.15.92:0/3967583593 wait complete. 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.110 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:09 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:09 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:10.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:10.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:10.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:09 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:10.489 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T21:01:10.491 INFO:tasks.cephadm:Deploying osd.8 on smithi142 with /dev/vg_nvme/lv_2... 2024-05-10T21:01:10.491 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T21:01:10.737 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:01:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:10 smithi092 ceph-mon[34415]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:10 smithi092 ceph-mon[34415]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:11.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:11.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:10 smithi142 ceph-mon[34846]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:11.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:10 smithi142 ceph-mon[34846]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:11.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:11.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:10 smithi050 ceph-mon[29546]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:11.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:10 smithi050 ceph-mon[29546]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:11 smithi092 ceph-mon[34415]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:12.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T21:01:12.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T21:01:12.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:12.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:11 smithi050 ceph-mon[29546]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:12.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T21:01:12.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: from='osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T21:01:12.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:12.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:01:12.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:11 smithi142 ceph-mon[34846]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:12.497 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T21:01:12.497 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T21:01:12.497 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T21:01:12.497 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T21:01:12.498 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114973 s, 912 MB/s 2024-05-10T21:01:12.498 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T21:01:13.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-05-10T21:01:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:12 smithi092 ceph-mon[34415]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:01:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:12 smithi092 ceph-mon[34415]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T21:01:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:12 smithi092 ceph-mon[34415]: from='osd.7 ' entity='osd.7' 2024-05-10T21:01:13.125 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:01:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:01:12.817+0000 7fc782ecc700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T21:01:13.126 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:01:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:01:12.933+0000 7fc77ac79700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T21:01:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:12 smithi142 ceph-mon[34846]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:01:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:12 smithi142 ceph-mon[34846]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T21:01:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:12 smithi142 ceph-mon[34846]: from='osd.7 ' entity='osd.7' 2024-05-10T21:01:13.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:12 smithi050 ceph-mon[29546]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:01:13.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:12 smithi050 ceph-mon[29546]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T21:01:13.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:13.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:12 smithi050 ceph-mon[29546]: from='osd.7 ' entity='osd.7' 2024-05-10T21:01:13.335 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi092 2024-05-10T21:01:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:13 smithi142 ceph-mon[34846]: osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] boot 2024-05-10T21:01:14.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:01:14.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi092 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:14.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:14.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:13 smithi050 ceph-mon[29546]: osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] boot 2024-05-10T21:01:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:01:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:01:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T21:01:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi092 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:14.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:13 smithi092 ceph-mon[34415]: osd.7 [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] boot 2024-05-10T21:01:14.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.977+0000 7ff811a6a700 1 -- 172.21.15.142:0/3611675747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 msgr2=0x7ff80c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:14.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.977+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3611675747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c104780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009990 tx=0x7ff7f4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.977+0000 7ff811a6a700 1 -- 172.21.15.142:0/3611675747 shutdown_connections 2024-05-10T21:01:14.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.977+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3611675747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c104d10 0x7ff80c10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3611675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c103240 0x7ff80c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3611675747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c104780 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 -- 172.21.15.142:0/3611675747 >> 172.21.15.142:0/3611675747 conn(0x7ff80c0fe880 msgr2=0x7ff80c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:01:14.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 -- 172.21.15.142:0/3611675747 shutdown_connections 2024-05-10T21:01:14.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 -- 172.21.15.142:0/3611675747 wait complete. 2024-05-10T21:01:14.985 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:14 smithi142 ceph-mon[34846]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T21:01:14.985 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:14 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:14.985 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:14 smithi142 ceph-mon[34846]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T21:01:14.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.978+0000 7ff811a6a700 1 Processor -- start 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 -- start start 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c103240 0x7ff80c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 0x7ff80c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c10ed20 con 0x7ff80c104d10 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c10eba0 con 0x7ff80c103240 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff811a6a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c10eea0 con 0x7ff80c104370 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39902/0 (socket says 172.21.15.142:39902) 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 -- 172.21.15.142:0/3025070984 learned_addr learned my addr 172.21.15.142:0/3025070984 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff80b7fe700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c103240 0x7ff80c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:14.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff80bfff700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 0x7ff80c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 -- 172.21.15.142:0/3025070984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c103240 msgr2=0x7ff80c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c103240 0x7ff80c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 -- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 msgr2=0x7ff80c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 0x7ff80c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff803fff700 1 -- 172.21.15.142:0/3025070984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c1a13c0 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.979+0000 7ff80bfff700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 0x7ff80c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff803fff700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40038c0 tx=0x7ff7f40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4025030 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c06b610 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f4021da0 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4005310 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.980+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c06bb70 con 0x7ff80c104370 2024-05-10T21:01:14.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.981+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c0512c0 con 0x7ff80c104370 2024-05-10T21:01:14.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.982+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff7f40049e0 con 0x7ff80c104370 2024-05-10T21:01:14.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.983+0000 7ff8097fa700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 0x7ff7f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:14.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.983+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7ff7f40e6df0 con 0x7ff80c104370 2024-05-10T21:01:14.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.983+0000 7ff80b7fe700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 0x7ff7f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:14.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.984+0000 7ff80b7fe700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 0x7ff7f80b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff80c104270 tx=0x7ff7fc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:01:14.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:14.987+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7f4013070 con 0x7ff80c104370 2024-05-10T21:01:15.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:14 smithi050 ceph-mon[29546]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T21:01:15.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:14 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:15.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:14 smithi050 ceph-mon[29546]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T21:01:15.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:15.199+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff80c06be90 con 0x7ff7f80b2880 2024-05-10T21:01:15.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:14 smithi092 ceph-mon[34415]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T21:01:15.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:14 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T21:01:15.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:14 smithi092 ceph-mon[34415]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:16.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:15 smithi142 ceph-mon[34846]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T21:01:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:16.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:16.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:16.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:15 smithi050 ceph-mon[29546]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:16.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:15 smithi092 ceph-mon[34415]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/3709474122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]': finished 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T21:01:18.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/3709474122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]': finished 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T21:01:18.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:18.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:18.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/3709474122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]: dispatch 2024-05-10T21:01:18.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b945a33a-16a3-4281-a58f-7a066f04c7e6"}]': finished 2024-05-10T21:01:18.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T21:01:18.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:19.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/2850851272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:19.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/2850851272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:19.926 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/2850851272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:20.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:20 smithi092 ceph-mon[34415]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:20 smithi142 ceph-mon[34846]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:20.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:20 smithi050 ceph-mon[29546]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:22 smithi092 ceph-mon[34415]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T21:01:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:22 smithi142 ceph-mon[34846]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T21:01:22.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:22 smithi050 ceph-mon[29546]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-10T21:01:24.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:24 smithi092 ceph-mon[34415]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:24 smithi142 ceph-mon[34846]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:24 smithi050 ceph-mon[29546]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:25.868 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T21:01:25.868 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:25.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T21:01:25.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:25.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T21:01:25.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:26.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:26 smithi092 ceph-mon[34415]: Deploying daemon osd.8 on smithi142 2024-05-10T21:01:26.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:26 smithi092 ceph-mon[34415]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T21:01:26.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:26 smithi142 ceph-mon[34846]: Deploying daemon osd.8 on smithi142 2024-05-10T21:01:26.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:26 smithi142 ceph-mon[34846]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T21:01:26.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:26 smithi050 ceph-mon[29546]: Deploying daemon osd.8 on smithi142 2024-05-10T21:01:26.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:26 smithi050 ceph-mon[29546]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-10T21:01:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:28 smithi092 ceph-mon[34415]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:28.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:28 smithi142 ceph-mon[34846]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:28 smithi050 ceph-mon[29546]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:30.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:30 smithi142 ceph-mon[34846]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T21:01:30.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:30 smithi092 ceph-mon[34415]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T21:01:30.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:30 smithi050 ceph-mon[29546]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-10T21:01:32.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:31 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:32.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:31 smithi050 ceph-mon[29546]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T21:01:32.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:31 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:31 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:31 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:32.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:31 smithi092 ceph-mon[34415]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T21:01:32.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:31 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:31 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:32.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:31 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:32.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:31 smithi142 ceph-mon[34846]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T21:01:34.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.787+0000 7ff8097fa700 1 -- 172.21.15.142:0/3025070984 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff80c06be90 con 0x7ff7f80b2880 2024-05-10T21:01:34.794 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 8 on host 'smithi142' 2024-05-10T21:01:34.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 msgr2=0x7ff7f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:34.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 0x7ff7f80b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff80c104270 tx=0x7ff7fc009190 comp rx=0 tx=0).stop 2024-05-10T21:01:34.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 msgr2=0x7ff80c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40038c0 tx=0x7ff7f40047b0 comp rx=0 tx=0).stop 2024-05-10T21:01:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 shutdown_connections 2024-05-10T21:01:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7f80b2880 0x7ff7f80b4d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff80c103240 0x7ff80c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff80c104d10 0x7ff80c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:34.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 --2- 172.21.15.142:0/3025070984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff80c104370 0x7ff80c199da0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:34.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 >> 172.21.15.142:0/3025070984 conn(0x7ff80c0fe880 msgr2=0x7ff80c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:01:34.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 shutdown_connections 2024-05-10T21:01:34.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:01:34.790+0000 7ff811a6a700 1 -- 172.21.15.142:0/3025070984 wait complete. 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:35.416 DEBUG:teuthology.orchestra.run.smithi142:osd.8> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T21:01:35.459 INFO:tasks.cephadm:Deploying osd.9 on smithi050 with /dev/vg_nvme/lv_1... 2024-05-10T21:01:35.460 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T21:01:35.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:35 smithi050 ceph-mon[29546]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:35.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:35 smithi050 ceph-mon[29546]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:35.944 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:01:36.102 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:35 smithi142 ceph-mon[34846]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:36.102 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:35 smithi142 ceph-mon[34846]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:36.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:35 smithi092 ceph-mon[34415]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:36.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:35 smithi092 ceph-mon[34415]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:37.068 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:36 smithi142 ceph-mon[34846]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:37.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:36 smithi092 ceph-mon[34415]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.145 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:37.145 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T21:01:37.146 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: from='osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.146 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T21:01:37.146 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:37.146 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:36 smithi050 ceph-mon[29546]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:01:37.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:37 smithi142 ceph-mon[34846]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:01:37.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:37 smithi142 ceph-mon[34846]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T21:01:37.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:37 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:37.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:37 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:37.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:37 smithi142 ceph-mon[34846]: from='osd.8 ' entity='osd.8' 2024-05-10T21:01:37.887 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:01:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:01:37.699+0000 7f37128b0700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T21:01:37.887 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:01:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:01:37.770+0000 7f370a65d700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T21:01:38.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:37 smithi092 ceph-mon[34415]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:01:38.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:37 smithi092 ceph-mon[34415]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T21:01:38.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:37 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:38.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:37 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:38.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:37 smithi092 ceph-mon[34415]: from='osd.8 ' entity='osd.8' 2024-05-10T21:01:38.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:37 smithi050 ceph-mon[29546]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:01:38.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:37 smithi050 ceph-mon[29546]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T21:01:38.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:37 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:38.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:37 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:38.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:37 smithi050 ceph-mon[29546]: from='osd.8 ' entity='osd.8' 2024-05-10T21:01:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T21:01:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T21:01:38.838 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-10T21:01:38.838 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-10T21:01:38.838 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.038427 s, 273 MB/s 2024-05-10T21:01:38.838 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] boot 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:39.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:38 smithi092 ceph-mon[34415]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] boot 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:39.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:39.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:38 smithi142 ceph-mon[34846]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: osd.8 [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] boot 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: osdmap e54: 9 total, 9 up, 9 in 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:01:39.176 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:38 smithi050 ceph-mon[29546]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T21:01:39.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-05-10T21:01:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:39 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi142 2024-05-10T21:01:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:39 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-10T21:01:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:39 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi142 2024-05-10T21:01:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:39 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-10T21:01:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:39 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi142 2024-05-10T21:01:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:39 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-10T21:01:40.219 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:01:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:40 smithi092 ceph-mon[34415]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:41.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:40 smithi092 ceph-mon[34415]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T21:01:41.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:40 smithi142 ceph-mon[34846]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:41.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:40 smithi142 ceph-mon[34846]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T21:01:41.162 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:40 smithi050 ceph-mon[29546]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:41.162 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:40 smithi050 ceph-mon[29546]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T21:01:42.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:41 smithi142 ceph-mon[34846]: pgmap v219: 1 pgs: 1 activating+degraded; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:42.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:41 smithi142 ceph-mon[34846]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T21:01:42.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:41 smithi050 ceph-mon[29546]: pgmap v219: 1 pgs: 1 activating+degraded; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:42.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:41 smithi050 ceph-mon[29546]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T21:01:42.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:41 smithi092 ceph-mon[34415]: pgmap v219: 1 pgs: 1 activating+degraded; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:42.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:41 smithi092 ceph-mon[34415]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T21:01:42.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 -- 172.21.15.50:0/1188445899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 msgr2=0x7f915010c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 --2- 172.21.15.50:0/1188445899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f915010c520 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f914800b0a0 tx=0x7f914800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 -- 172.21.15.50:0/1188445899 shutdown_connections 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 --2- 172.21.15.50:0/1188445899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f915010c520 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 --2- 172.21.15.50:0/1188445899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150104e40 0x7f9150107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 --2- 172.21.15.50:0/1188445899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501077a0 0x7f9150109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 -- 172.21.15.50:0/1188445899 >> 172.21.15.50:0/1188445899 conn(0x7f91500fe880 msgr2=0x7f9150100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 -- 172.21.15.50:0/1188445899 shutdown_connections 2024-05-10T21:01:42.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.449+0000 7f9155800700 1 -- 172.21.15.50:0/1188445899 wait complete. 2024-05-10T21:01:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.450+0000 7f9155800700 1 Processor -- start 2024-05-10T21:01:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.450+0000 7f9155800700 1 -- start start 2024-05-10T21:01:42.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 0x7f915019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f91501a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f915010fde0 con 0x7f915010a100 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f915010fc60 con 0x7f9150104e40 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f9155800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f915010ff60 con 0x7f91501077a0 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f914ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f914ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44266/0 (socket says 172.21.15.50:44266) 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f914ea00700 1 -- 172.21.15.50:0/1914758669 learned_addr learned my addr 172.21.15.50:0/1914758669 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f914f400700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f91501a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:42.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.451+0000 7f914e000700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 0x7f915019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 -- 172.21.15.50:0/1914758669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 msgr2=0x7f915019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 0x7f915019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 -- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 msgr2=0x7f91501a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:01:42.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f91501a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 -- 172.21.15.50:0/1914758669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91501a4640 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914e000700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 0x7f915019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f914ea00700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f914000e970 tx=0x7f914000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914000cbd0 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f914000cd70 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140005680 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.452+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91501a4930 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.453+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501a4e70 con 0x7f9150104e40 2024-05-10T21:01:42.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.454+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91500512c0 con 0x7f9150104e40 2024-05-10T21:01:42.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.456+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9140004750 con 0x7f9150104e40 2024-05-10T21:01:42.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.458+0000 7f9147400700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 0x7f91340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:01:42.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.458+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f91400e1cc0 con 0x7f9150104e40 2024-05-10T21:01:42.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.458+0000 7f914e000700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 0x7f91340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:01:42.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.459+0000 7f914e000700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 0x7f91340b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f915019db90 tx=0x7f913c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:01:42.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.462+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91400e8080 con 0x7f9150104e40 2024-05-10T21:01:42.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:01:42.718+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9150103340 con 0x7f91340b2880 2024-05-10T21:01:43.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:42 smithi142 ceph-mon[34846]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:43.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:43.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:43.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:42 smithi050 ceph-mon[29546]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:43.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:43.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:42 smithi092 ceph-mon[34415]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:01:43.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:01:43.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:01:43.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:44.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:43 smithi050 ceph-mon[29546]: pgmap v220: 1 pgs: 1 activating+degraded; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:44.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:43 smithi092 ceph-mon[34415]: pgmap v220: 1 pgs: 1 activating+degraded; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:44.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:43 smithi142 ceph-mon[34846]: pgmap v220: 1 pgs: 1 activating+degraded; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T21:01:45.714 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:45 smithi050 ceph-mon[29546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T21:01:45.714 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:45 smithi050 ceph-mon[29546]: Cluster is now healthy 2024-05-10T21:01:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:45 smithi092 ceph-mon[34415]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T21:01:45.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:45 smithi092 ceph-mon[34415]: Cluster is now healthy 2024-05-10T21:01:45.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:45 smithi142 ceph-mon[34846]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T21:01:45.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:45 smithi142 ceph-mon[34846]: Cluster is now healthy 2024-05-10T21:01:46.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:46 smithi092 ceph-mon[34415]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T21:01:46.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:46 smithi142 ceph-mon[34846]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T21:01:46.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:46 smithi050 ceph-mon[29546]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T21:01:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:47 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1821856555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:47 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:47 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]': finished 2024-05-10T21:01:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:47 smithi092 ceph-mon[34415]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T21:01:47.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:01:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:47 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1821856555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:47 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:47 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]': finished 2024-05-10T21:01:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:47 smithi142 ceph-mon[34846]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T21:01:47.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:01:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:47 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1821856555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:47 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]: dispatch 2024-05-10T21:01:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:47 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "462243ae-aa14-4706-828a-a0d7247e5afc"}]': finished 2024-05-10T21:01:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:47 smithi050 ceph-mon[29546]: osdmap e57: 10 total, 9 up, 10 in 2024-05-10T21:01:47.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:01:48.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:48 smithi092 ceph-mon[34415]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:48.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:48 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1380145153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:48 smithi142 ceph-mon[34846]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:48.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:48 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1380145153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:48.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:48 smithi050 ceph-mon[29546]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T21:01:48.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:48 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1380145153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:01:50.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:50 smithi142 ceph-mon[34846]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T21:01:50.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:50 smithi050 ceph-mon[29546]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T21:01:51.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:50 smithi092 ceph-mon[34415]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T21:01:52.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:52 smithi142 ceph-mon[34846]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:52.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:52 smithi050 ceph-mon[29546]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:53.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:52 smithi092 ceph-mon[34415]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:54.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:54 smithi050 ceph-mon[29546]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:54.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:54 smithi142 ceph-mon[34846]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:55.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:54 smithi092 ceph-mon[34415]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:01:56.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:56 smithi050 ceph-mon[29546]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:56.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T21:01:56.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:57.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:56 smithi092 ceph-mon[34415]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:57.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T21:01:57.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:57.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:56 smithi142 ceph-mon[34846]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:57.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T21:01:57.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:01:57.873 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:57 smithi050 ceph-mon[29546]: Deploying daemon osd.9 on smithi050 2024-05-10T21:01:58.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:57 smithi092 ceph-mon[34415]: Deploying daemon osd.9 on smithi050 2024-05-10T21:01:58.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:57 smithi142 ceph-mon[34846]: Deploying daemon osd.9 on smithi050 2024-05-10T21:01:58.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:01:58 smithi050 ceph-mon[29546]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:01:58 smithi092 ceph-mon[34415]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:01:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:01:58 smithi142 ceph-mon[34846]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:00.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:00 smithi050 ceph-mon[29546]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:01.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:00 smithi092 ceph-mon[34415]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:01.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:00 smithi142 ceph-mon[34846]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:02.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:02 smithi050 ceph-mon[29546]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:02 smithi092 ceph-mon[34415]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:03.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:02 smithi142 ceph-mon[34846]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:04.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:04 smithi050 ceph-mon[29546]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:05.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:04 smithi092 ceph-mon[34415]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:05.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:04 smithi142 ceph-mon[34846]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:06 smithi050 ceph-mon[29546]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:06.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:07.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:06 smithi092 ceph-mon[34415]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:07.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:07.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:07.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:07.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:06 smithi142 ceph-mon[34846]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:08.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:08 smithi050 ceph-mon[29546]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:08.938 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:08 smithi142 ceph-mon[34846]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:09.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:08 smithi092 ceph-mon[34415]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:10 smithi092 ceph-mon[34415]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:10 smithi092 ceph-mon[34415]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:10 smithi092 ceph-mon[34415]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:10 smithi142 ceph-mon[34846]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:11.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:10 smithi142 ceph-mon[34846]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:10 smithi142 ceph-mon[34846]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:10 smithi050 ceph-mon[29546]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:11.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:11.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:10 smithi050 ceph-mon[29546]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:10 smithi050 ceph-mon[29546]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T21:02:11.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.680+0000 7f9147400700 1 -- 172.21.15.50:0/1914758669 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9150103340 con 0x7f91340b2880 2024-05-10T21:02:11.688 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 9 on host 'smithi050' 2024-05-10T21:02:11.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.684+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 msgr2=0x7f91340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:11.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.684+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 0x7f91340b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f915019db90 tx=0x7f913c009260 comp rx=0 tx=0).stop 2024-05-10T21:02:11.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.684+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 msgr2=0x7f915019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:11.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.684+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f914000e970 tx=0x7f914000ec80 comp rx=0 tx=0).stop 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 shutdown_connections 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91340b2880 0x7f91340b4d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150104e40 0x7f915019cae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f915010a100 0x7f91501a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 --2- 172.21.15.50:0/1914758669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91501077a0 0x7f915019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 >> 172.21.15.50:0/1914758669 conn(0x7f91500fe880 msgr2=0x7f9150102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 shutdown_connections 2024-05-10T21:02:11.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:02:11.685+0000 7f9155800700 1 -- 172.21.15.50:0/1914758669 wait complete. 2024-05-10T21:02:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T21:02:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T21:02:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:12.126 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:11 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:12.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:11 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:12.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:11 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:12.508 DEBUG:teuthology.orchestra.run.smithi050:osd.9> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T21:02:12.513 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-05-10T21:02:12.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T21:02:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:02:13.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:12 smithi050 ceph-mon[29546]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:02:13.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:12 smithi050 ceph-mon[29546]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T21:02:13.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:13.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:12 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:12 smithi142 ceph-mon[34846]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:02:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:12 smithi142 ceph-mon[34846]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T21:02:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:13.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:12 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:13.219 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:12 smithi092 ceph-mon[34415]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-10T21:02:13.219 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:12 smithi092 ceph-mon[34415]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T21:02:13.219 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:13.219 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:12 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] boot 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.026 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:13 smithi050 ceph-mon[29546]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] boot 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:13 smithi142 ceph-mon[34846]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:14.204 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: osd.9 [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] boot 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: osdmap e60: 10 total, 10 up, 10 in 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T21:02:14.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:13 smithi092 ceph-mon[34415]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-10T21:02:14.773 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129987 s, 807 MB/s 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 2024-05-10T21:02:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-10T21:02:15.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:14 smithi092 ceph-mon[34415]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T21:02:15.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:14 smithi142 ceph-mon[34846]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T21:02:15.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:14 smithi050 ceph-mon[29546]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T21:02:15.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-05-10T21:02:15.683 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:02:15.988 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:15 smithi092 ceph-mon[34415]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:16.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:15 smithi142 ceph-mon[34846]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:16.397 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:15 smithi050 ceph-mon[29546]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 -- 172.21.15.92:0/3238244567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81029e0 msgr2=0x7f73a8102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 --2- 172.21.15.92:0/3238244567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81029e0 0x7f73a8102df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7398009990 tx=0x7f7398009ca0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 -- 172.21.15.92:0/3238244567 shutdown_connections 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 --2- 172.21.15.92:0/3238244567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81029e0 0x7f73a8102df0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 --2- 172.21.15.92:0/3238244567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a81044b0 0x7f73a810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 --2- 172.21.15.92:0/3238244567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a8103b10 0x7f73a8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.330+0000 7f73afb45700 1 -- 172.21.15.92:0/3238244567 >> 172.21.15.92:0/3238244567 conn(0x7f73a80fe020 msgr2=0x7f73a81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.331+0000 7f73afb45700 1 -- 172.21.15.92:0/3238244567 shutdown_connections 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.331+0000 7f73afb45700 1 -- 172.21.15.92:0/3238244567 wait complete. 2024-05-10T21:02:17.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.331+0000 7f73afb45700 1 Processor -- start 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.331+0000 7f73afb45700 1 -- start start 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a81029e0 0x7f73a80fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a8103b10 0x7f73a819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810e520 con 0x7f73a81029e0 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810e3a0 con 0x7f73a81044b0 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73afb45700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810e6a0 con 0x7f73a8103b10 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45072/0 (socket says 172.21.15.92:45072) 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 -- 172.21.15.92:0/2373213413 learned_addr learned my addr 172.21.15.92:0/2373213413 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ad0e0700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a8103b10 0x7f73a819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 -- 172.21.15.92:0/2373213413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a8103b10 msgr2=0x7f73a819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a8103b10 0x7f73a819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ad8e1700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a81029e0 0x7f73a80fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 -- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a81029e0 msgr2=0x7f73a80fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a81029e0 0x7f73a80fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 -- 172.21.15.92:0/2373213413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a81a2610 con 0x7f73a81044b0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f73ae0e2700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b970 tx=0x7f73a000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.332+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0019070 con 0x7f73a81044b0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.333+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a81a2900 con 0x7f73a81044b0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.333+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a000cde0 con 0x7f73a81044b0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.333+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0007370 con 0x7f73a81044b0 2024-05-10T21:02:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.333+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a2e70 con 0x7f73a81044b0 2024-05-10T21:02:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.334+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a80512c0 con 0x7f73a81044b0 2024-05-10T21:02:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.335+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f73a001c070 con 0x7f73a81044b0 2024-05-10T21:02:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.336+0000 7f739effd700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 0x7f73940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:17.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.336+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f73a00e16a0 con 0x7f73a81044b0 2024-05-10T21:02:17.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.336+0000 7f73ad8e1700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 0x7f73940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:17.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.338+0000 7f73ad8e1700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 0x7f73940b4d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f739800b5c0 tx=0x7f7398003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:02:17.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.339+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73a0023070 con 0x7f73a81044b0 2024-05-10T21:02:17.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:17.552+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f73a8063c00 con 0x7f73940b2880 2024-05-10T21:02:18.592 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi050 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:18.593 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:18.594 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:18.594 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi050 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:18.637 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi050 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-10T21:02:18.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:18.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:20.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:20.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/173597878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]': finished 2024-05-10T21:02:20.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T21:02:20.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/173597878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]': finished 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T21:02:20.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:20.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:20.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/173597878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]: dispatch 2024-05-10T21:02:20.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08409a23-50ab-4268-9ad6-c50879b535f7"}]': finished 2024-05-10T21:02:20.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: osdmap e62: 11 total, 10 up, 11 in 2024-05-10T21:02:20.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:21.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:21 smithi092 ceph-mon[34415]: from='client.? 172.21.15.92:0/3399471292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:21.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:21 smithi142 ceph-mon[34846]: from='client.? 172.21.15.92:0/3399471292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:21.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:21 smithi050 ceph-mon[29546]: from='client.? 172.21.15.92:0/3399471292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:22.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:22 smithi092 ceph-mon[34415]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:22.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:22 smithi142 ceph-mon[34846]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:22.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:22 smithi050 ceph-mon[29546]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:24.835 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:24 smithi092 ceph-mon[34415]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:24.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:24 smithi142 ceph-mon[34846]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:24.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:24 smithi050 ceph-mon[29546]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:26.877 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:26 smithi092 ceph-mon[34415]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:26.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:26 smithi142 ceph-mon[34846]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:26.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:26 smithi050 ceph-mon[29546]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:28.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:28 smithi142 ceph-mon[34846]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:28.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T21:02:28.886 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:28 smithi050 ceph-mon[29546]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T21:02:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:28.935 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:28 smithi092 ceph-mon[34415]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:28.935 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T21:02:28.935 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:29.813 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:29 smithi092 ceph-mon[34415]: Deploying daemon osd.10 on smithi092 2024-05-10T21:02:29.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:29 smithi050 ceph-mon[29546]: Deploying daemon osd.10 on smithi092 2024-05-10T21:02:30.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:29 smithi142 ceph-mon[34846]: Deploying daemon osd.10 on smithi092 2024-05-10T21:02:30.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:30 smithi092 ceph-mon[34415]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:30.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:30 smithi050 ceph-mon[29546]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:31.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:30 smithi142 ceph-mon[34846]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:32.728 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:32 smithi092 ceph-mon[34415]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:32.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:32 smithi050 ceph-mon[29546]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:33.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:32 smithi142 ceph-mon[34846]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:34 smithi050 ceph-mon[29546]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:34.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:34 smithi092 ceph-mon[34415]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:34 smithi142 ceph-mon[34846]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:35.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.015 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:37.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:37.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:37.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:36 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:37.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:36 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:37.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:36 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:37.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.443+0000 7f739effd700 1 -- 172.21.15.92:0/2373213413 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f73a8063c00 con 0x7f73940b2880 2024-05-10T21:02:37.450 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.445+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 msgr2=0x7f73940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.445+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 0x7f73940b4d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f739800b5c0 tx=0x7f7398003860 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.445+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 msgr2=0x7f73a81a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.445+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73a000b970 tx=0x7f73a000bc80 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.445+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 shutdown_connections 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f73940b2880 0x7f73940b4d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73a81044b0 0x7f73a81a1de0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73a81029e0 0x7f73a80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 --2- 172.21.15.92:0/2373213413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a8103b10 0x7f73a819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 >> 172.21.15.92:0/2373213413 conn(0x7f73a80fe020 msgr2=0x7f73a8107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 shutdown_connections 2024-05-10T21:02:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-10T21:02:37.446+0000 7f73afb45700 1 -- 172.21.15.92:0/2373213413 wait complete. 2024-05-10T21:02:38.163 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T21:02:38.165 INFO:tasks.cephadm:Deploying osd.11 on smithi142 with /dev/vg_nvme/lv_1... 2024-05-10T21:02:38.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T21:02:38.413 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:02:38.473 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.473 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:38.473 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:38.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:38.474 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:38 smithi092 ceph-mon[34415]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:38.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:38.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:38.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:38.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:38 smithi142 ceph-mon[34846]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:38.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:38.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:38.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:38.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:38.675 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:38 smithi050 ceph-mon[29546]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T21:02:39.555 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:39 smithi092 ceph-mon[34415]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T21:02:39.555 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:39 smithi092 ceph-mon[34415]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:39.556 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:39 smithi092 ceph-mon[34415]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T21:02:39.556 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:39.556 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:39 smithi092 ceph-mon[34415]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:39.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:39 smithi142 ceph-mon[34846]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T21:02:39.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:39 smithi142 ceph-mon[34846]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:39.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:39 smithi142 ceph-mon[34846]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T21:02:39.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:39.636 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:39 smithi142 ceph-mon[34846]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:39.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:39 smithi050 ceph-mon[29546]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T21:02:39.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:39 smithi050 ceph-mon[29546]: from='osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:39.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:39 smithi050 ceph-mon[29546]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T21:02:39.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:39.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:39 smithi050 ceph-mon[29546]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116699 s, 899 MB/s 2024-05-10T21:02:40.567 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:40.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:40 smithi092 ceph-mon[34415]: from='osd.10 ' entity='osd.10' 2024-05-10T21:02:40.626 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:02:40 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:02:40.329+0000 7fcc9ed72700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T21:02:40.626 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:02:40 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:02:40.523+0000 7fcc9631e700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:40.665 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:40 smithi142 ceph-mon[34846]: from='osd.10 ' entity='osd.10' 2024-05-10T21:02:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-10T21:02:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T21:02:40.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:40.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:40.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:40 smithi050 ceph-mon[29546]: from='osd.10 ' entity='osd.10' 2024-05-10T21:02:41.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-05-10T21:02:41.422 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.728 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:41.729 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:41.729 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.729 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] boot 2024-05-10T21:02:41.729 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T21:02:41.729 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:41 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] boot 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T21:02:41.876 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:41 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:41.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:41.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: osd.10 [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] boot 2024-05-10T21:02:41.926 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T21:02:41.926 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:41 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T21:02:42.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:42 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi092 2024-05-10T21:02:42.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:42 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-10T21:02:42.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:42 smithi142 ceph-mon[34846]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:42.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:42 smithi142 ceph-mon[34846]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T21:02:42.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 -- 172.21.15.142:0/2515773862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 msgr2=0x7f05dc104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 --2- 172.21.15.142:0/2515773862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc104790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009990 tx=0x7f05cc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 -- 172.21.15.142:0/2515773862 shutdown_connections 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 --2- 172.21.15.142:0/2515773862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc104d20 0x7f05dc10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 --2- 172.21.15.142:0/2515773862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05dc103250 0x7f05dc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.902+0000 7f05e0e54700 1 --2- 172.21.15.142:0/2515773862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc104790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.903+0000 7f05e0e54700 1 -- 172.21.15.142:0/2515773862 >> 172.21.15.142:0/2515773862 conn(0x7f05dc0fe890 msgr2=0x7f05dc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.903+0000 7f05e0e54700 1 -- 172.21.15.142:0/2515773862 shutdown_connections 2024-05-10T21:02:42.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.903+0000 7f05e0e54700 1 -- 172.21.15.142:0/2515773862 wait complete. 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.903+0000 7f05e0e54700 1 Processor -- start 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.903+0000 7f05e0e54700 1 -- start start 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc103250 0x7f05dc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05dc104d20 0x7f05dc1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10ed30 con 0x7f05dc104d20 2024-05-10T21:02:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10ebb0 con 0x7f05dc103250 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10eeb0 con 0x7f05dc104380 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48320/0 (socket says 172.21.15.142:48320) 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 -- 172.21.15.142:0/3330842143 learned_addr learned my addr 172.21.15.142:0/3330842143 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05da59c700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc103250 0x7f05dc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 -- 172.21.15.142:0/3330842143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc103250 msgr2=0x7f05dc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc103250 0x7f05dc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 -- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05dc104d20 msgr2=0x7f05dc1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05dc104d20 0x7f05dc1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 -- 172.21.15.142:0/3330842143 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc1a13b0 con 0x7f05dc104380 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05d9d9b700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003800 tx=0x7f05cc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc004830 con 0x7f05dc104380 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc1a1640 con 0x7f05dc104380 2024-05-10T21:02:42.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05cc005020 con 0x7f05dc104380 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc003e40 con 0x7f05dc104380 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.904+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1a1b10 con 0x7f05dc104380 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.905+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc0512c0 con 0x7f05dc104380 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.907+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f05cc029060 con 0x7f05dc104380 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.907+0000 7f05cb7fe700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 0x7f05c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:02:42.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.907+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f05cc0e2aa0 con 0x7f05dc104380 2024-05-10T21:02:42.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.908+0000 7f05da59c700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 0x7f05c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:02:42.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.909+0000 7f05da59c700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 0x7f05c40b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05dc104280 tx=0x7f05d0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:02:42.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:42.911+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05cc013070 con 0x7f05dc104380 2024-05-10T21:02:43.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:42 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi092 2024-05-10T21:02:43.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:42 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-10T21:02:43.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:42 smithi092 ceph-mon[34415]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:43.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:42 smithi092 ceph-mon[34415]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T21:02:43.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:02:43.123+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f05dc063c00 con 0x7f05c40b2880 2024-05-10T21:02:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:42 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi092 2024-05-10T21:02:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:42 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-10T21:02:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:42 smithi050 ceph-mon[29546]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-10T21:02:43.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:42 smithi050 ceph-mon[29546]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T21:02:44.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:43 smithi142 ceph-mon[34846]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T21:02:44.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:44.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:44.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:43 smithi050 ceph-mon[29546]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T21:02:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:43 smithi092 ceph-mon[34415]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T21:02:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T21:02:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T21:02:44.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:44.837 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:44 smithi142 ceph-mon[34846]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:44.837 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:44 smithi142 ceph-mon[34846]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:45.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:44 smithi050 ceph-mon[29546]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:45.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:44 smithi050 ceph-mon[29546]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:45.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:44 smithi092 ceph-mon[34415]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:02:45.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:44 smithi092 ceph-mon[34415]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:46.087 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:45 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/3901565495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.087 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:45 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.087 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:45 smithi142 ceph-mon[34846]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]': finished 2024-05-10T21:02:46.087 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:45 smithi142 ceph-mon[34846]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T21:02:46.087 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:02:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:45 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/3901565495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:45 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:45 smithi092 ceph-mon[34415]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]': finished 2024-05-10T21:02:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:45 smithi092 ceph-mon[34415]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T21:02:46.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:02:46.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:45 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/3901565495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:45 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]: dispatch 2024-05-10T21:02:46.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:45 smithi050 ceph-mon[29546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6ba8a0f-61ce-48d4-a456-18eca77cbdcf"}]': finished 2024-05-10T21:02:46.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:45 smithi050 ceph-mon[29546]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T21:02:46.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:02:47.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:46 smithi092 ceph-mon[34415]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T21:02:47.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:46 smithi092 ceph-mon[34415]: from='client.? 172.21.15.142:0/236205109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:46 smithi142 ceph-mon[34846]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T21:02:47.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:46 smithi142 ceph-mon[34846]: from='client.? 172.21.15.142:0/236205109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:47.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:46 smithi050 ceph-mon[29546]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-10T21:02:47.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:46 smithi050 ceph-mon[29546]: from='client.? 172.21.15.142:0/236205109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T21:02:49.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:48 smithi092 ceph-mon[34415]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T21:02:49.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:48 smithi142 ceph-mon[34846]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T21:02:49.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:48 smithi050 ceph-mon[29546]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T21:02:51.048 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:50 smithi142 ceph-mon[34846]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T21:02:51.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:50 smithi092 ceph-mon[34415]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T21:02:51.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:50 smithi050 ceph-mon[29546]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-10T21:02:53.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:52 smithi092 ceph-mon[34415]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:02:53.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:52 smithi142 ceph-mon[34846]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:02:53.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:52 smithi050 ceph-mon[29546]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-10T21:02:54.105 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T21:02:54.105 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:54.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T21:02:54.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T21:02:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:02:54.994 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:54 smithi142 ceph-mon[34846]: Deploying daemon osd.11 on smithi142 2024-05-10T21:02:54.994 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:54 smithi142 ceph-mon[34846]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:02:55.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:54 smithi092 ceph-mon[34415]: Deploying daemon osd.11 on smithi142 2024-05-10T21:02:55.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:54 smithi092 ceph-mon[34415]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:02:55.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:54 smithi050 ceph-mon[29546]: Deploying daemon osd.11 on smithi142 2024-05-10T21:02:55.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:54 smithi050 ceph-mon[29546]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T21:02:57.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:56 smithi092 ceph-mon[34415]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:57.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:56 smithi142 ceph-mon[34846]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:57.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:56 smithi050 ceph-mon[29546]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:59.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:58 smithi092 ceph-mon[34415]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:02:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:58 smithi142 ceph-mon[34846]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:02:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:02:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:58 smithi050 ceph-mon[29546]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:02:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:02:59.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:02:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:00.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:00 smithi092 ceph-mon[34415]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:01.117 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:00 smithi142 ceph-mon[34846]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:01.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:00 smithi050 ceph-mon[29546]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:01.866 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 11 on host 'smithi142' 2024-05-10T21:03:01.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.860+0000 7f05cb7fe700 1 -- 172.21.15.142:0/3330842143 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f05dc063c00 con 0x7f05c40b2880 2024-05-10T21:03:01.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.862+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 msgr2=0x7f05c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:01.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.862+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 0x7f05c40b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05dc104280 tx=0x7f05d0009190 comp rx=0 tx=0).stop 2024-05-10T21:03:01.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.862+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 msgr2=0x7f05dc199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:01.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.862+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003800 tx=0x7f05cc0047b0 comp rx=0 tx=0).stop 2024-05-10T21:03:01.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 shutdown_connections 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f05c40b2880 0x7f05c40b4d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05dc103250 0x7f05dc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05dc104d20 0x7f05dc1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 --2- 172.21.15.142:0/3330842143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05dc104380 0x7f05dc199d90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 >> 172.21.15.142:0/3330842143 conn(0x7f05dc0fe890 msgr2=0x7f05dc1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 shutdown_connections 2024-05-10T21:03:01.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:01.863+0000 7f05e0e54700 1 -- 172.21.15.142:0/3330842143 wait complete. 2024-05-10T21:03:02.448 DEBUG:teuthology.orchestra.run.smithi142:osd.11> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T21:03:02.450 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T21:03:02.450 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T21:03:02.823 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:03:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:03:02.500+0000 7f723f43a7c0 -1 osd.11 0 log_to_monitors true 2024-05-10T21:03:02.913 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T21:03:03.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:03 smithi142 ceph-mon[34846]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T21:03:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:03 smithi142 ceph-mon[34846]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T21:03:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:03 smithi142 ceph-mon[34846]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:03.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:03 smithi142 ceph-mon[34846]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:04.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:03 smithi050 ceph-mon[29546]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T21:03:04.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:03 smithi050 ceph-mon[29546]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T21:03:04.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:04.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:03 smithi050 ceph-mon[29546]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:04.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:03 smithi050 ceph-mon[29546]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:04.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:03 smithi092 ceph-mon[34415]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T21:03:04.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:03 smithi092 ceph-mon[34415]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T21:03:04.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:04.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:03 smithi092 ceph-mon[34415]: from='osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:04.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:03 smithi092 ceph-mon[34415]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-10T21:03:04.864 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:03:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:03:04.795+0000 7f7234a9e700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 -- 172.21.15.50:0/454140676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010c420 msgr2=0x7f96d0112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 --2- 172.21.15.50:0/454140676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010c420 0x7f96d0112cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009f90 tx=0x7f96c800b330 comp rx=0 tx=0).stop 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 -- 172.21.15.50:0/454140676 shutdown_connections 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 --2- 172.21.15.50:0/454140676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010c420 0x7f96d0112cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 --2- 172.21.15.50:0/454140676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010ab10 0x7f96d010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 --2- 172.21.15.50:0/454140676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 0x7f96d010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 -- 172.21.15.50:0/454140676 >> 172.21.15.50:0/454140676 conn(0x7f96d00799c0 msgr2=0x7f96d007be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 -- 172.21.15.50:0/454140676 shutdown_connections 2024-05-10T21:03:05.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.111+0000 7f96d5e00700 1 -- 172.21.15.50:0/454140676 wait complete. 2024-05-10T21:03:05.119 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:05.119 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:03:05.119 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T21:03:05.120 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.120 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.120 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:04 smithi050 ceph-mon[29546]: from='osd.11 ' entity='osd.11' 2024-05-10T21:03:05.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.112+0000 7f96d5e00700 1 Processor -- start 2024-05-10T21:03:05.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.112+0000 7f96d5e00700 1 -- start start 2024-05-10T21:03:05.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:05.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 0x7f96d01a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 0x7f96d01a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0116410 con 0x7f96d010ba80 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0116290 con 0x7f96d010ab10 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96d5e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0116590 con 0x7f96d010c420 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96cea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96cea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57654/0 (socket says 172.21.15.50:57654) 2024-05-10T21:03:05.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96cea00700 1 -- 172.21.15.50:0/3315788264 learned_addr learned my addr 172.21.15.50:0/3315788264 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:05.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96cf400700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 0x7f96d01a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:05.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.113+0000 7f96ce000700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 0x7f96d01a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:05.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 -- 172.21.15.50:0/3315788264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 msgr2=0x7f96d01a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:05.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 0x7f96d01a83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 -- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 msgr2=0x7f96d01a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 0x7f96d01a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 -- 172.21.15.50:0/3315788264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d01a8a20 con 0x7f96d010ab10 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cf400700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 0x7f96d01a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96cea00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96c000e940 tx=0x7f96c000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0004220 con 0x7f96d010ab10 2024-05-10T21:03:05.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c00043c0 con 0x7f96d010ab10 2024-05-10T21:03:05.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d01a8d10 con 0x7f96d010ab10 2024-05-10T21:03:05.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.114+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0003cb0 con 0x7f96d010ab10 2024-05-10T21:03:05.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.115+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01a91e0 con 0x7f96d010ab10 2024-05-10T21:03:05.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.116+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d00512c0 con 0x7f96d010ab10 2024-05-10T21:03:05.128 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:05.129 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:03:05.129 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T21:03:05.130 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.131 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.131 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:04 smithi092 ceph-mon[34415]: from='osd.11 ' entity='osd.11' 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.118+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f96c000c7c0 con 0x7f96d010ab10 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.120+0000 7f96d4a00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 0x7f96b40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.120+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f96c00e1e30 con 0x7f96d010ab10 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.120+0000 7f96ce000700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 0x7f96b40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.121+0000 7f96ce000700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 0x7f96b40b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96d01a1f70 tx=0x7f96b8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.122+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96c00aa070 con 0x7f96d010ab10 2024-05-10T21:03:05.135 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:03:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:03:04.957+0000 7f722c84b700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T21:03:05.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:05.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-10T21:03:05.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T21:03:05.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:05.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:04 smithi142 ceph-mon[34846]: from='osd.11 ' entity='osd.11' 2024-05-10T21:03:05.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.365+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f96d002e5f0 con 0x7f96d010ab10 2024-05-10T21:03:05.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.366+0000 7f96d4a00700 1 -- 172.21.15.50:0/3315788264 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f96d002e5f0 con 0x7f96d010ab10 2024-05-10T21:03:05.373 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:05.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 msgr2=0x7f96b40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:05.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 0x7f96b40b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96d01a1f70 tx=0x7f96b8009310 comp rx=0 tx=0).stop 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 msgr2=0x7f96d01a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96c000e940 tx=0x7f96c000ec50 comp rx=0 tx=0).stop 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 shutdown_connections 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96b40b2900 0x7f96b40b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d010ab10 0x7f96d01a0ec0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d010c420 0x7f96d01a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 --2- 172.21.15.50:0/3315788264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96d010ba80 0x7f96d01a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 >> 172.21.15.50:0/3315788264 conn(0x7f96d00799c0 msgr2=0x7f96d010ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 shutdown_connections 2024-05-10T21:03:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:05.370+0000 7f96d5e00700 1 -- 172.21.15.50:0/3315788264 wait complete. 2024-05-10T21:03:06.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:05 smithi050 ceph-mon[29546]: purged_snaps scrub starts 2024-05-10T21:03:06.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:05 smithi050 ceph-mon[29546]: purged_snaps scrub ok 2024-05-10T21:03:06.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:06.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:05 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3315788264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:06.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:06.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:05 smithi142 ceph-mon[34846]: purged_snaps scrub starts 2024-05-10T21:03:06.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:05 smithi142 ceph-mon[34846]: purged_snaps scrub ok 2024-05-10T21:03:06.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:06.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:05 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3315788264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:06.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:06.257 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715374961,"num_in_osds":12,"osd_in_since":1715374965,"num_remapped_pgs":0} 2024-05-10T21:03:06.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:05 smithi092 ceph-mon[34415]: purged_snaps scrub starts 2024-05-10T21:03:06.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:05 smithi092 ceph-mon[34415]: purged_snaps scrub ok 2024-05-10T21:03:06.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:06.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:05 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3315788264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:06.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:07.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] boot 2024-05-10T21:03:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T21:03:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: Detected new or changed devices on smithi142 2024-05-10T21:03:07.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.137 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-10T21:03:07.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:07.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] boot 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: Detected new or changed devices on smithi142 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.175 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.259 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T21:03:07.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: osd.11 [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] boot 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: Detected new or changed devices on smithi142 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:07.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:07.710 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:08.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:08 smithi050 ceph-mon[29546]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T21:03:08.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:08 smithi050 ceph-mon[29546]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:08.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:08 smithi092 ceph-mon[34415]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T21:03:08.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:08 smithi092 ceph-mon[34415]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:08 smithi142 ceph-mon[34846]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T21:03:08.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:08 smithi142 ceph-mon[34846]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:10.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.070+0000 7f3c9da00700 1 -- 172.21.15.50:0/1792223550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ba80 msgr2=0x7f3c9810be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.070+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1792223550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ba80 0x7f3c9810be90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009990 tx=0x7f3c80009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 -- 172.21.15.50:0/1792223550 shutdown_connections 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1792223550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ab10 0x7f3c9810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1792223550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 0x7f3c98112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1792223550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ba80 0x7f3c9810be90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 -- 172.21.15.50:0/1792223550 >> 172.21.15.50:0/1792223550 conn(0x7f3c980799c0 msgr2=0x7f3c9807be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 -- 172.21.15.50:0/1792223550 shutdown_connections 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.071+0000 7f3c9da00700 1 -- 172.21.15.50:0/1792223550 wait complete. 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.072+0000 7f3c9da00700 1 Processor -- start 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.072+0000 7f3c9da00700 1 -- start start 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:10.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 0x7f3c981a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 0x7f3c981a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981163c0 con 0x7f3c9810c420 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98116240 con 0x7f3c9810ba80 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c9da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98116540 con 0x7f3c9810ab10 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47760/0 (socket says 172.21.15.50:47760) 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 -- 172.21.15.50:0/1452687383 learned_addr learned my addr 172.21.15.50:0/1452687383 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c97400700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 0x7f3c981a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:10.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c8e000700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 0x7f3c981a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 -- 172.21.15.50:0/1452687383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 msgr2=0x7f3c981a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 0x7f3c981a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 -- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 msgr2=0x7f3c981a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 0x7f3c981a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.073+0000 7f3c96a00700 1 -- 172.21.15.50:0/1452687383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c981a8a40 con 0x7f3c9810ab10 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c8e000700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 0x7f3c981a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c96a00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ef70 tx=0x7f3c8800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c880092e0 con 0x7f3c9810ab10 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c88003a00 con 0x7f3c9810ab10 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88004580 con 0x7f3c9810ab10 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c981a8cd0 con 0x7f3c9810ab10 2024-05-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.074+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981a91b0 con 0x7f3c9810ab10 2024-05-10T21:03:10.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.076+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c980512c0 con 0x7f3c9810ab10 2024-05-10T21:03:10.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.078+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3c8801b030 con 0x7f3c9810ab10 2024-05-10T21:03:10.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.079+0000 7f3c8fe00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 0x7f3c7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:10.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.079+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f3c880e1c70 con 0x7f3c9810ab10 2024-05-10T21:03:10.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.079+0000 7f3c8e000700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 0x7f3c7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:10.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.080+0000 7f3c8e000700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 0x7f3c7c0b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b6c0 tx=0x7f3c80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:10.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.083+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c88022020 con 0x7f3c9810ab10 2024-05-10T21:03:10.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.315+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3c980643d0 con 0x7f3c9810ab10 2024-05-10T21:03:10.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.316+0000 7f3c8fe00700 1 -- 172.21.15.50:0/1452687383 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f3c88010040 con 0x7f3c9810ab10 2024-05-10T21:03:10.322 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.318+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 msgr2=0x7f3c7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.318+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 0x7f3c7c0b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b6c0 tx=0x7f3c80005fb0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.318+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 msgr2=0x7f3c981a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.318+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ef70 tx=0x7f3c8800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 shutdown_connections 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3c7c0b2900 0x7f3c7c0b4e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c9810c420 0x7f3c981a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9810ab10 0x7f3c981a0ee0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 --2- 172.21.15.50:0/1452687383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c9810ba80 0x7f3c981a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 >> 172.21.15.50:0/1452687383 conn(0x7f3c980799c0 msgr2=0x7f3c9810ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 shutdown_connections 2024-05-10T21:03:10.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:10.319+0000 7f3c9da00700 1 -- 172.21.15.50:0/1452687383 wait complete. 2024-05-10T21:03:10.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:10 smithi050 ceph-mon[29546]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:10.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:10 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1452687383' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:10.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:10 smithi092 ceph-mon[34415]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:10.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:10 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1452687383' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:10.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:10 smithi142 ceph-mon[34846]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:10.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:10 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1452687383' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T21:03:11.107 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1715374985,"num_in_osds":12,"osd_in_since":1715374965,"num_remapped_pgs":0} 2024-05-10T21:03:11.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:03:11.567 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:12.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:12 smithi092 ceph-mon[34415]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:12.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:12 smithi142 ceph-mon[34846]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:12.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:12 smithi050 ceph-mon[29546]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:13.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 -- 172.21.15.50:0/2977471049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d010a990 msgr2=0x7fd5d010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:13.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2977471049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d010a990 0x7fd5d010ada0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009990 tx=0x7fd5bc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 -- 172.21.15.50:0/2977471049 shutdown_connections 2024-05-10T21:03:13.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2977471049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010b2e0 0x7fd5d0117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2977471049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d0110970 0x7fd5d0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2977471049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d010a990 0x7fd5d010ada0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 -- 172.21.15.50:0/2977471049 >> 172.21.15.50:0/2977471049 conn(0x7fd5d0077c80 msgr2=0x7fd5d00780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 -- 172.21.15.50:0/2977471049 shutdown_connections 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.824+0000 7fd5d7600700 1 -- 172.21.15.50:0/2977471049 wait complete. 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.826+0000 7fd5d7600700 1 Processor -- start 2024-05-10T21:03:13.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.826+0000 7fd5d7600700 1 -- start start 2024-05-10T21:03:13.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 0x7fd5d01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:13.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 0x7fd5d01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:13.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d0110970 0x7fd5d01a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:13.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d011aae0 con 0x7fd5d010b2e0 2024-05-10T21:03:13.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d011a960 con 0x7fd5d0110970 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d7600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d011ac60 con 0x7fd5d010a990 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5cfe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 0x7fd5d01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 0x7fd5d01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5cfe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 0x7fd5d01a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32958/0 (socket says 172.21.15.50:32958) 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5cfe00700 1 -- 172.21.15.50:0/2703542279 learned_addr learned my addr 172.21.15.50:0/2703542279 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:13.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.827+0000 7fd5d5600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d0110970 0x7fd5d01a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 -- 172.21.15.50:0/2703542279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d0110970 msgr2=0x7fd5d01a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d0110970 0x7fd5d01a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 -- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 msgr2=0x7fd5d01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 0x7fd5d01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 -- 172.21.15.50:0/2703542279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01a6340 con 0x7fd5d010a990 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d4c00700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 0x7fd5d01a1510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9c0 tx=0x7fd5c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:13.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400c9c0 con 0x7fd5d010a990 2024-05-10T21:03:13.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c400cb60 con 0x7fd5d010a990 2024-05-10T21:03:13.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4018680 con 0x7fd5d010a990 2024-05-10T21:03:13.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.828+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01a6630 con 0x7fd5d010a990 2024-05-10T21:03:13.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.829+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01adf50 con 0x7fd5d010a990 2024-05-10T21:03:13.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.830+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d010c080 con 0x7fd5d010a990 2024-05-10T21:03:13.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.833+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd5c4010430 con 0x7fd5d010a990 2024-05-10T21:03:13.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.834+0000 7fd5cd600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 0x7fd5b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:13.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.834+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd5c40e1f10 con 0x7fd5d010a990 2024-05-10T21:03:13.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.834+0000 7fd5cfe00700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 0x7fd5b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:13.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.835+0000 7fd5cfe00700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 0x7fd5b80b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0109870 tx=0x7fd5bc00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:13.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:13.838+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5c40aa030 con 0x7fd5d010a990 2024-05-10T21:03:14.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.083+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5d002e5f0 con 0x7fd5d010a990 2024-05-10T21:03:14.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.085+0000 7fd5cd600700 1 -- 172.21.15.50:0/2703542279 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18407 (secure 0 0 0) 0x7fd5c40b20b0 con 0x7fd5d010a990 2024-05-10T21:03:14.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:14.091 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":73,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:03:07.520299+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:03:14.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.088+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 msgr2=0x7fd5b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:14.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.088+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 0x7fd5b80b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0109870 tx=0x7fd5bc00b280 comp rx=0 tx=0).stop 2024-05-10T21:03:14.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.088+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 msgr2=0x7fd5d01a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:14.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.088+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 0x7fd5d01a1510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9c0 tx=0x7fd5c400ecd0 comp rx=0 tx=0).stop 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 shutdown_connections 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd5b80b2880 0x7fd5b80b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d0110970 0x7fd5d01a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5d010a990 0x7fd5d01a1510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 --2- 172.21.15.50:0/2703542279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5d010b2e0 0x7fd5d01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 >> 172.21.15.50:0/2703542279 conn(0x7fd5d0077c80 msgr2=0x7fd5d007b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 shutdown_connections 2024-05-10T21:03:14.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:14.089+0000 7fd5d7600700 1 -- 172.21.15.50:0/2703542279 wait complete. 2024-05-10T21:03:14.868 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:14 smithi050 ceph-mon[29546]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:14.868 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:14 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2703542279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:03:14.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:14 smithi092 ceph-mon[34415]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:14.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:14 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2703542279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:03:14.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:14 smithi142 ceph-mon[34846]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:14.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:14 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2703542279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:03:14.926 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T20:58:45.564017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T21:03:14.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T21:03:15.398 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:16.819 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:16 smithi050 ceph-mon[29546]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:16.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:16 smithi092 ceph-mon[34415]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:16.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:16 smithi142 ceph-mon[34846]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:17.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 -- 172.21.15.50:0/1660735169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c076450 msgr2=0x7fcf8c1106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 --2- 172.21.15.50:0/1660735169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c076450 0x7fcf8c1106c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009990 tx=0x7fcf78009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 -- 172.21.15.50:0/1660735169 shutdown_connections 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 --2- 172.21.15.50:0/1660735169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c110c00 0x7fcf8c113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 --2- 172.21.15.50:0/1660735169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 --2- 172.21.15.50:0/1660735169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c076450 0x7fcf8c1106c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 -- 172.21.15.50:0/1660735169 >> 172.21.15.50:0/1660735169 conn(0x7fcf8c100040 msgr2=0x7fcf8c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 -- 172.21.15.50:0/1660735169 shutdown_connections 2024-05-10T21:03:17.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.697+0000 7fcf91e00700 1 -- 172.21.15.50:0/1660735169 wait complete. 2024-05-10T21:03:17.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.698+0000 7fcf91e00700 1 Processor -- start 2024-05-10T21:03:17.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 -- start start 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c076450 0x7fcf8c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c116710 con 0x7fcf8c110c00 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c116590 con 0x7fcf8c076450 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf91e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c116890 con 0x7fcf8c075b00 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf8b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf8a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c076450 0x7fcf8c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf8b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57668/0 (socket says 172.21.15.50:57668) 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf8b400700 1 -- 172.21.15.50:0/3141897992 learned_addr learned my addr 172.21.15.50:0/3141897992 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:17.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.699+0000 7fcf8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 -- 172.21.15.50:0/3141897992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c076450 msgr2=0x7fcf8c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c076450 0x7fcf8c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 -- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 msgr2=0x7fcf8c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 -- 172.21.15.50:0/3141897992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c1a1f70 con 0x7fcf8c075b00 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8b400700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.700+0000 7fcf8aa00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c19d0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00e9f0 tx=0x7fcf7c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.701+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00c9c0 con 0x7fcf8c075b00 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.701+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c00cb60 con 0x7fcf8c075b00 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.701+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c1a2260 con 0x7fcf8c075b00 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.701+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c018680 con 0x7fcf8c075b00 2024-05-10T21:03:17.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.701+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c1a9b80 con 0x7fcf8c075b00 2024-05-10T21:03:17.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.702+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c0512c0 con 0x7fcf8c075b00 2024-05-10T21:03:17.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.704+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcf7c010430 con 0x7fcf8c075b00 2024-05-10T21:03:17.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.706+0000 7fcf90a00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 0x7fcf700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:17.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.706+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fcf7c0e22e0 con 0x7fcf8c075b00 2024-05-10T21:03:17.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.706+0000 7fcf8a000700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 0x7fcf700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:17.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.707+0000 7fcf8a000700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 0x7fcf700b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c103490 tx=0x7fcf7800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:17.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.708+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf7c0aa400 con 0x7fcf8c075b00 2024-05-10T21:03:17.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.947+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf8c02e5f0 con 0x7fcf8c075b00 2024-05-10T21:03:17.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.948+0000 7fcf90a00700 1 -- 172.21.15.50:0/3141897992 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fcf7c0a9d00 con 0x7fcf8c075b00 2024-05-10T21:03:17.953 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 msgr2=0x7fcf700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 0x7fcf700b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c103490 tx=0x7fcf7800b260 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 msgr2=0x7fcf8c19d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c19d0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00e9f0 tx=0x7fcf7c00ed00 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 shutdown_connections 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcf700b2880 0x7fcf700b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf8c110c00 0x7fcf8c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf8c075b00 0x7fcf8c19d0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 --2- 172.21.15.50:0/3141897992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf8c076450 0x7fcf8c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 >> 172.21.15.50:0/3141897992 conn(0x7fcf8c100040 msgr2=0x7fcf8c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 shutdown_connections 2024-05-10T21:03:17.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:17.952+0000 7fcf91e00700 1 -- 172.21.15.50:0/3141897992 wait complete. 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.a on smithi050 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.d on smithi050 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.g on smithi050 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.j on smithi050 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-05-10T21:03:18.711 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-05-10T21:03:18.712 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-05-10T21:03:18.712 INFO:tasks.cephadm:Adding mds.c on smithi142 2024-05-10T21:03:18.712 INFO:tasks.cephadm:Adding mds.f on smithi142 2024-05-10T21:03:18.712 INFO:tasks.cephadm:Adding mds.i on smithi142 2024-05-10T21:03:18.712 INFO:tasks.cephadm:Adding mds.l on smithi142 2024-05-10T21:03:18.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l' 2024-05-10T21:03:18.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:18 smithi092 ceph-mon[34415]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:18.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:18 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3141897992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:03:18.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:18 smithi142 ceph-mon[34846]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:18.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:18 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3141897992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:03:18.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:18 smithi050 ceph-mon[29546]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:18.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:18 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3141897992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:03:18.961 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 -- 172.21.15.142:0/3525788307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 msgr2=0x7f0e5c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3525788307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c104780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009990 tx=0x7f0e44009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 -- 172.21.15.142:0/3525788307 shutdown_connections 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3525788307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c103240 0x7f0e5c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3525788307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e5c104d10 0x7f0e5c10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3525788307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c104780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 -- 172.21.15.142:0/3525788307 >> 172.21.15.142:0/3525788307 conn(0x7f0e5c0fe880 msgr2=0x7f0e5c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 -- 172.21.15.142:0/3525788307 shutdown_connections 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.634+0000 7f0e61b74700 1 -- 172.21.15.142:0/3525788307 wait complete. 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 Processor -- start 2024-05-10T21:03:20.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 -- start start 2024-05-10T21:03:20.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e5c103240 0x7f0e5c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:20.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:20.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 0x7f0e5c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10ed20 con 0x7f0e5c103240 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10eba0 con 0x7f0e5c104d10 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e61b74700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10eea0 con 0x7f0e5c104370 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e52ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e52ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47040/0 (socket says 172.21.15.142:47040) 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e52ffd700 1 -- 172.21.15.142:0/3511032386 learned_addr learned my addr 172.21.15.142:0/3511032386 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.635+0000 7f0e5bfff700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 0x7f0e5c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 -- 172.21.15.142:0/3511032386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 msgr2=0x7f0e5c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:20.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 -- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e5c103240 msgr2=0x7f0e5c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e5c103240 0x7f0e5c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 -- 172.21.15.142:0/3511032386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c1a1370 con 0x7f0e5c104d10 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e52ffd700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e5bfff700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 0x7f0e5c1a0cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400be20 tx=0x7f0e54009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54019070 con 0x7f0e5c104d10 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5400cc10 con 0x7f0e5c104d10 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e540125a0 con 0x7f0e5c104d10 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c1a1660 con 0x7f0e5c104d10 2024-05-10T21:03:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.636+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c1a1bd0 con 0x7f0e5c104d10 2024-05-10T21:03:20.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.637+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5c0512c0 con 0x7f0e5c104d10 2024-05-10T21:03:20.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.638+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e5401c070 con 0x7f0e5c104d10 2024-05-10T21:03:20.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.639+0000 7f0e597fa700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 0x7f0e480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:20.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.639+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f0e540e1440 con 0x7f0e5c104d10 2024-05-10T21:03:20.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.640+0000 7f0e5b7fe700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 0x7f0e480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:20.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.640+0000 7f0e5b7fe700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 0x7f0e480b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c104270 tx=0x7f0e4c0096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:20.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.643+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e540a9560 con 0x7f0e5c104d10 2024-05-10T21:03:20.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.854+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}) -- 0x7f0e5c063e00 con 0x7f0e480b2880 2024-05-10T21:03:20.860 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:20 smithi142 ceph-mon[34846]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:20.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:20 smithi092 ceph-mon[34415]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:20.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.872+0000 7f0e597fa700 1 -- 172.21.15.142:0/3511032386 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0e5c063e00 con 0x7f0e480b2880 2024-05-10T21:03:20.878 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mds.all update... 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 msgr2=0x7f0e480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 0x7f0e480b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c104270 tx=0x7f0e4c0096c0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 msgr2=0x7f0e5c1a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 0x7f0e5c1a0cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400be20 tx=0x7f0e54009ed0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 shutdown_connections 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e480b2880 0x7f0e480b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c104d10 0x7f0e5c1a0cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e5c103240 0x7f0e5c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 --2- 172.21.15.142:0/3511032386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e5c104370 0x7f0e5c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:20.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.874+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 >> 172.21.15.142:0/3511032386 conn(0x7f0e5c0fe880 msgr2=0x7f0e5c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:20.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.875+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 shutdown_connections 2024-05-10T21:03:20.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-10T21:03:20.875+0000 7f0e61b74700 1 -- 172.21.15.142:0/3511032386 wait complete. 2024-05-10T21:03:20.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:20 smithi050 ceph-mon[29546]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:21.698 DEBUG:teuthology.orchestra.run.smithi050:mds.a> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T21:03:21.703 DEBUG:teuthology.orchestra.run.smithi050:mds.d> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T21:03:21.751 DEBUG:teuthology.orchestra.run.smithi050:mds.g> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T21:03:21.797 DEBUG:teuthology.orchestra.run.smithi050:mds.j> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T21:03:21.842 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T21:03:21.844 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T21:03:21.888 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T21:03:21.931 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T21:03:21.974 DEBUG:teuthology.orchestra.run.smithi142:mds.c> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T21:03:21.976 DEBUG:teuthology.orchestra.run.smithi142:mds.f> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T21:03:21.995 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:21 smithi142 ceph-mon[34846]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:03:21.995 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:21 smithi142 ceph-mon[34846]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-05-10T21:03:21.995 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:21.995 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:21 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:21.996 DEBUG:teuthology.orchestra.run.smithi142:mds.i> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T21:03:22.039 DEBUG:teuthology.orchestra.run.smithi142:mds.l> sudo journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T21:03:22.082 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T21:03:22.083 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:22.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:21 smithi092 ceph-mon[34415]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:03:22.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:21 smithi092 ceph-mon[34415]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-05-10T21:03:22.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:22.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:21 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:22.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:21 smithi050 ceph-mon[29546]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:03:22.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:21 smithi050 ceph-mon[29546]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-05-10T21:03:22.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:22.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:21 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:03:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:23.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:23.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:23.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:23.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:24.125 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:23 smithi142 ceph-mon[34846]: Deploying daemon mds.l on smithi142 2024-05-10T21:03:24.126 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:23 smithi142 ceph-mon[34846]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:24.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:23 smithi050 ceph-mon[29546]: Deploying daemon mds.l on smithi142 2024-05-10T21:03:24.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:23 smithi050 ceph-mon[29546]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:24.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:23 smithi092 ceph-mon[34415]: Deploying daemon mds.l on smithi142 2024-05-10T21:03:24.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:23 smithi092 ceph-mon[34415]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:24.385 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 systemd[1]: Starting Ceph mds.l for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:24.822 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 podman[64941]: 2024-05-10 21:03:24.423162972 +0000 UTC m=+0.017656956 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:24.823 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 podman[64941]: 2024-05-10 21:03:24.531881768 +0000 UTC m=+0.126375743 container create f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-10T21:03:24.823 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 podman[64941]: 2024-05-10 21:03:24.725724816 +0000 UTC m=+0.320218788 container init f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-10T21:03:24.823 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 podman[64941]: 2024-05-10 21:03:24.728323793 +0000 UTC m=+0.322817775 container start f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:03:24.823 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 ceph-mds[64960]: -- 172.21.15.142:0/1389185558 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a226ac3a40 con 0x55a226b0ba80 2024-05-10T21:03:24.823 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l[64956]: starting mds.l at 2024-05-10T21:03:24.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.967+0000 7fc882a00700 1 -- 172.21.15.50:0/327557962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 msgr2=0x7fc87c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.967+0000 7fc882a00700 1 --2- 172.21.15.50:0/327557962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c113200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b0a0 tx=0x7fc87400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 -- 172.21.15.50:0/327557962 shutdown_connections 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 --2- 172.21.15.50:0/327557962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c113200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 --2- 172.21.15.50:0/327557962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 0x7fc87c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 --2- 172.21.15.50:0/327557962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 0x7fc87c10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 -- 172.21.15.50:0/327557962 >> 172.21.15.50:0/327557962 conn(0x7fc87c102030 msgr2=0x7fc87c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 -- 172.21.15.50:0/327557962 shutdown_connections 2024-05-10T21:03:24.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.968+0000 7fc882a00700 1 -- 172.21.15.50:0/327557962 wait complete. 2024-05-10T21:03:24.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.970+0000 7fc882a00700 1 Processor -- start 2024-05-10T21:03:24.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.970+0000 7fc882a00700 1 -- start start 2024-05-10T21:03:24.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 0x7fc87c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:24.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 0x7fc87c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:24.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1164d0 con 0x7fc87c106620 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c116350 con 0x7fc87c10b8a0 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c116650 con 0x7fc87c10af50 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc881600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 0x7fc87c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41360/0 (socket says 172.21.15.50:41360) 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc882000700 1 -- 172.21.15.50:0/4039258944 learned_addr learned my addr 172.21.15.50:0/4039258944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:24.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.971+0000 7fc880c00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 0x7fc87c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:24.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 -- 172.21.15.50:0/4039258944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 msgr2=0x7fc87c19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:24.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 0x7fc87c19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 -- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 msgr2=0x7fc87c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:24.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 0x7fc87c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:24.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 -- 172.21.15.50:0/4039258944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a1cf0 con 0x7fc87c10b8a0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882000700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b780 tx=0x7fc874004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874021070 con 0x7fc87c10b8a0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.972+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a1f80 con 0x7fc87c10b8a0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.973+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a9900 con 0x7fc87c10b8a0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.973+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8740075c0 con 0x7fc87c10b8a0 2024-05-10T21:03:24.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.973+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87401d9f0 con 0x7fc87c10b8a0 2024-05-10T21:03:24.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.974+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c005330 con 0x7fc87c10b8a0 2024-05-10T21:03:24.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.978+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc874024070 con 0x7fc87c10b8a0 2024-05-10T21:03:24.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.979+0000 7fc87a000700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 0x7fc8640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:24.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.980+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc8740e2850 con 0x7fc87c10b8a0 2024-05-10T21:03:24.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.980+0000 7fc881600700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 0x7fc8640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.980+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc87402b030 con 0x7fc87c10b8a0 2024-05-10T21:03:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:24.981+0000 7fc881600700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 0x7fc8640b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc870009680 tx=0x7fc870009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:25.135 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 bash[64941]: f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 2024-05-10T21:03:25.135 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:03:24 smithi142 systemd[1]: Started Ceph mds.l for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:25.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.268+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc85c005620 con 0x7fc87c10b8a0 2024-05-10T21:03:25.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.269+0000 7fc87a000700 1 -- 172.21.15.50:0/4039258944 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc8740aaa00 con 0x7fc87c10b8a0 2024-05-10T21:03:25.275 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:25.275 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T21:03:25.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 msgr2=0x7fc8640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:25.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 0x7fc8640b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc870009680 tx=0x7fc870009260 comp rx=0 tx=0).stop 2024-05-10T21:03:25.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 msgr2=0x7fc87c1a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:25.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b780 tx=0x7fc874004d00 comp rx=0 tx=0).stop 2024-05-10T21:03:25.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 shutdown_connections 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8640b2880 0x7fc8640b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc87c10b8a0 0x7fc87c1a1650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc87c106620 0x7fc87c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 --2- 172.21.15.50:0/4039258944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c10af50 0x7fc87c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.274+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 >> 172.21.15.50:0/4039258944 conn(0x7fc87c102030 msgr2=0x7fc87c10a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.275+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 shutdown_connections 2024-05-10T21:03:25.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:25.275+0000 7fc882a00700 1 -- 172.21.15.50:0/4039258944 wait complete. 2024-05-10T21:03:25.282 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 2 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6832/2643414360,v1:172.21.15.142:6833/2643414360] up:boot 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: fsmap 1 up:standby 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:25.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:25 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4039258944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:25.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6832/2643414360,v1:172.21.15.142:6833/2643414360] up:boot 2024-05-10T21:03:25.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: fsmap 1 up:standby 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:25.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:25 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4039258944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6832/2643414360,v1:172.21.15.142:6833/2643414360] up:boot 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: fsmap 1 up:standby 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:26.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:25 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4039258944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:26.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:26 smithi092 ceph-mon[34415]: Deploying daemon mds.k on smithi092 2024-05-10T21:03:26.840 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:26 smithi092 ceph-mon[34415]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:27.124 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 systemd[1]: Starting Ceph mds.k for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:27.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:26 smithi142 ceph-mon[34846]: Deploying daemon mds.k on smithi092 2024-05-10T21:03:27.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:26 smithi142 ceph-mon[34846]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:27.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:26 smithi050 ceph-mon[29546]: Deploying daemon mds.k on smithi092 2024-05-10T21:03:27.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:26 smithi050 ceph-mon[29546]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:27.509 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 podman[63863]: 2024-05-10 21:03:27.232335843 +0000 UTC m=+0.114658091 container create d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:03:27.509 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 podman[63863]: 2024-05-10 21:03:27.135978389 +0000 UTC m=+0.018300648 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:27.509 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 podman[63863]: 2024-05-10 21:03:27.468473462 +0000 UTC m=+0.350795710 container init d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:03:27.509 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 podman[63863]: 2024-05-10 21:03:27.471205034 +0000 UTC m=+0.353527284 container start d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:03:27.874 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 ceph-mds[63882]: -- 172.21.15.92:0/4240014853 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed91c8da40 con 0x55ed91cd5180 2024-05-10T21:03:27.874 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k[63878]: starting mds.k at 2024-05-10T21:03:27.874 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 bash[63863]: d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 2024-05-10T21:03:27.874 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:03:27 smithi092 systemd[1]: Started Ceph mds.k for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:28.086 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:28.522 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:28.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: Deploying daemon mds.j on smithi050 2024-05-10T21:03:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6832/194068448,v1:172.21.15.92:6833/194068448] up:boot 2024-05-10T21:03:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: fsmap 2 up:standby 2024-05-10T21:03:28.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:28 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T21:03:28.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: Deploying daemon mds.j on smithi050 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6832/194068448,v1:172.21.15.92:6833/194068448] up:boot 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: fsmap 2 up:standby 2024-05-10T21:03:28.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:28 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:28.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:28.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: Deploying daemon mds.j on smithi050 2024-05-10T21:03:28.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6832/194068448,v1:172.21.15.92:6833/194068448] up:boot 2024-05-10T21:03:28.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: fsmap 2 up:standby 2024-05-10T21:03:28.925 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:28 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T21:03:30.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:30 smithi142 ceph-mon[34846]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:30.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:30 smithi050 ceph-mon[29546]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:31.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:30 smithi092 ceph-mon[34415]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 -- 172.21.15.50:0/2215724607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c1054e0 msgr2=0x7f0f2c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2215724607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c1058f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009990 tx=0x7f0f1c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 -- 172.21.15.50:0/2215724607 shutdown_connections 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2215724607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 0x7f0f2c110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2215724607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c1058f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2215724607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c105e30 0x7f0f2c10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 -- 172.21.15.50:0/2215724607 >> 172.21.15.50:0/2215724607 conn(0x7f0f2c0fedc0 msgr2=0x7f0f2c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:32.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 -- 172.21.15.50:0/2215724607 shutdown_connections 2024-05-10T21:03:32.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:31.999+0000 7f0f30e00700 1 -- 172.21.15.50:0/2215724607 wait complete. 2024-05-10T21:03:32.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.000+0000 7f0f30e00700 1 Processor -- start 2024-05-10T21:03:32.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 -- start start 2024-05-10T21:03:32.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:32.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c105e30 0x7f0f2c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:32.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 0x7f0f2c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c114480 con 0x7f0f2c105e30 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c114300 con 0x7f0f2c10e910 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.001+0000 7f0f30e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c114600 con 0x7f0f2c1054e0 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c105e30 0x7f0f2c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50234/0 (socket says 172.21.15.50:50234) 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 -- 172.21.15.50:0/2697325011 learned_addr learned my addr 172.21.15.50:0/2697325011 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f2a800700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 0x7f0f2c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 -- 172.21.15.50:0/2697325011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 msgr2=0x7f0f2c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 0x7f0f2c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 -- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c105e30 msgr2=0x7f0f2c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:32.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c105e30 0x7f0f2c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.002+0000 7f0f29e00700 1 -- 172.21.15.50:0/2697325011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c1a1f40 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.003+0000 7f0f29e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005910 tx=0x7f0f1c00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.003+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c027070 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.003+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1a21d0 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.003+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1a9b50 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.004+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1c022470 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.004+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c017630 con 0x7f0f2c1054e0 2024-05-10T21:03:32.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.005+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c005330 con 0x7f0f2c1054e0 2024-05-10T21:03:32.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.009+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0f1c02a070 con 0x7f0f2c1054e0 2024-05-10T21:03:32.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.011+0000 7f0f22a00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 0x7f0f100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:32.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.011+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f0f1c0e2ce0 con 0x7f0f2c1054e0 2024-05-10T21:03:32.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.011+0000 7f0f29400700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 0x7f0f100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:32.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.012+0000 7f0f29400700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 0x7f0f100b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c1022b0 tx=0x7f0f18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:32.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.015+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f1c05fdf0 con 0x7f0f2c1054e0 2024-05-10T21:03:32.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.337+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f0c005620 con 0x7f0f2c1054e0 2024-05-10T21:03:32.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.337+0000 7f0f22a00700 1 -- 172.21.15.50:0/2697325011 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f0f1c013070 con 0x7f0f2c1054e0 2024-05-10T21:03:32.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:32.343 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-10T21:03:32.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 msgr2=0x7f0f100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 0x7f0f100b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c1022b0 tx=0x7f0f18009260 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 msgr2=0x7f0f2c19d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005910 tx=0x7f0f1c00b670 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 shutdown_connections 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0f100b2900 0x7f0f100b4e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f2c10e910 0x7f0f2c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f2c1054e0 0x7f0f2c19d070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 --2- 172.21.15.50:0/2697325011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c105e30 0x7f0f2c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 >> 172.21.15.50:0/2697325011 conn(0x7f0f2c0fedc0 msgr2=0x7f0f2c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 shutdown_connections 2024-05-10T21:03:32.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:32.341+0000 7f0f30e00700 1 -- 172.21.15.50:0/2697325011 wait complete. 2024-05-10T21:03:32.348 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2024-05-10T21:03:32.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:32 smithi142 ceph-mon[34846]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:32.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:32 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2697325011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:32.901 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:32 smithi050 ceph-mon[29546]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:32.901 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:32 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2697325011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:33.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:32 smithi092 ceph-mon[34415]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:33.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:32 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2697325011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:33.173 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:03:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j[67837]: starting mds.j at 2024-05-10T21:03:33.673 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:03:33 smithi050 bash[67809]: dfc2796d73f317d26bd3a5479d7deb18bb2c80021bca3a6dca4e64212a3cb403 2024-05-10T21:03:33.674 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:03:33 smithi050 systemd[1]: Started Ceph mds.j for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6834/1323665657,v1:172.21.15.50:6835/1323665657] up:boot 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: fsmap 3 up:standby 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:34.889 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:34.890 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:34.890 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:34.890 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:34 smithi050 ceph-mon[29546]: Deploying daemon mds.d on smithi050 2024-05-10T21:03:35.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:35.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6834/1323665657,v1:172.21.15.50:6835/1323665657] up:boot 2024-05-10T21:03:35.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: fsmap 3 up:standby 2024-05-10T21:03:35.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:35.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:34 smithi092 ceph-mon[34415]: Deploying daemon mds.d on smithi050 2024-05-10T21:03:35.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6834/1323665657,v1:172.21.15.50:6835/1323665657] up:boot 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: fsmap 3 up:standby 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:35.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:34 smithi142 ceph-mon[34846]: Deploying daemon mds.d on smithi050 2024-05-10T21:03:35.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:36.146 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:36.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:36 smithi050 ceph-mon[29546]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:37.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:36 smithi092 ceph-mon[34415]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:37.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:36 smithi142 ceph-mon[34846]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:38.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.690+0000 7f838dc00700 1 -- 172.21.15.50:0/1697409563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880764b0 msgr2=0x7f8388114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.690+0000 7f838dc00700 1 --2- 172.21.15.50:0/1697409563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880764b0 0x7f8388114e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8374009990 tx=0x7f8374009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 -- 172.21.15.50:0/1697409563 shutdown_connections 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 --2- 172.21.15.50:0/1697409563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8388115370 0x7f8388117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 --2- 172.21.15.50:0/1697409563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 0x7f8388075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 --2- 172.21.15.50:0/1697409563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83880764b0 0x7f8388114e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 -- 172.21.15.50:0/1697409563 >> 172.21.15.50:0/1697409563 conn(0x7f8388100040 msgr2=0x7f8388102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 -- 172.21.15.50:0/1697409563 shutdown_connections 2024-05-10T21:03:38.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.691+0000 7f838dc00700 1 -- 172.21.15.50:0/1697409563 wait complete. 2024-05-10T21:03:38.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.692+0000 7f838dc00700 1 Processor -- start 2024-05-10T21:03:38.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.692+0000 7f838dc00700 1 -- start start 2024-05-10T21:03:38.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 0x7f83881a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 0x7f83881a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 0x7f83881a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f838811aac0 con 0x7f83880764b0 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f838811a940 con 0x7f8388115370 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f838dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f838811ac40 con 0x7f8388075b60 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f8386000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 0x7f83881a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f8387400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 0x7f83881a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f8386000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 0x7f83881a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48598/0 (socket says 172.21.15.50:48598) 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f8386000700 1 -- 172.21.15.50:0/3083543881 learned_addr learned my addr 172.21.15.50:0/3083543881 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:38.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.693+0000 7f8386a00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 0x7f83881a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:38.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 -- 172.21.15.50:0/3083543881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 msgr2=0x7f83881a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:38.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 0x7f83881a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 -- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 msgr2=0x7f83881a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:38.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 0x7f83881a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 -- 172.21.15.50:0/3083543881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83881a6320 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.694+0000 7f8387400700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 0x7f83881a5c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f838000efc0 tx=0x7f838000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.695+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800092e0 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.695+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83881a65b0 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.695+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8380004bb0 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.695+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380004020 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.695+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83881adf30 con 0x7f8388115370 2024-05-10T21:03:38.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.696+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83880772a0 con 0x7f8388115370 2024-05-10T21:03:38.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.698+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f838001b030 con 0x7f8388115370 2024-05-10T21:03:38.702 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:38 smithi050 ceph-mon[29546]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:38.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.700+0000 7f837f400700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 0x7f836c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:38.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.701+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f83800e1920 con 0x7f8388115370 2024-05-10T21:03:38.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.701+0000 7f8386a00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 0x7f836c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:38.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.702+0000 7f8386a00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 0x7f836c0b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8378009c60 tx=0x7f8378009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:38.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.702+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8380022020 con 0x7f8388115370 2024-05-10T21:03:38.937 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:38 smithi142 ceph-mon[34846]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:38.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.992+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83881a27b0 con 0x7f8388115370 2024-05-10T21:03:38.997 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:03:38 smithi050 systemd[1]: Started Ceph mds.d for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:38.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.993+0000 7f837f400700 1 -- 172.21.15.50:0/3083543881 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f8380010040 con 0x7f8388115370 2024-05-10T21:03:38.998 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:38.998 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-10T21:03:39.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.996+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 msgr2=0x7f836c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:39.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.996+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 0x7f836c0b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8378009c60 tx=0x7f8378009260 comp rx=0 tx=0).stop 2024-05-10T21:03:39.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.996+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 msgr2=0x7f83881a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:39.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.996+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 0x7f83881a5c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f838000efc0 tx=0x7f838000c3a0 comp rx=0 tx=0).stop 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 shutdown_connections 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f836c0b2900 0x7f836c0b4e10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388115370 0x7f83881a5c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8388075b60 0x7f83881a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 --2- 172.21.15.50:0/3083543881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83880764b0 0x7f83881a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 >> 172.21.15.50:0/3083543881 conn(0x7f8388100040 msgr2=0x7f8388100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 shutdown_connections 2024-05-10T21:03:39.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:38.997+0000 7f838dc00700 1 -- 172.21.15.50:0/3083543881 wait complete. 2024-05-10T21:03:39.002 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 4 2024-05-10T21:03:39.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:38 smithi092 ceph-mon[34415]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:40.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3083543881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6836/1369162577,v1:172.21.15.50:6837/1369162577] up:boot 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: fsmap 4 up:standby 2024-05-10T21:03:40.125 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:39 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:40.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:40.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3083543881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:40.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6836/1369162577,v1:172.21.15.50:6837/1369162577] up:boot 2024-05-10T21:03:40.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: fsmap 4 up:standby 2024-05-10T21:03:40.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:39 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:40.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3083543881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6836/1369162577,v1:172.21.15.50:6837/1369162577] up:boot 2024-05-10T21:03:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: fsmap 4 up:standby 2024-05-10T21:03:40.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:39 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T21:03:41.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:40 smithi092 ceph-mon[34415]: Deploying daemon mds.e on smithi092 2024-05-10T21:03:41.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:40 smithi092 ceph-mon[34415]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:41.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:40 smithi142 ceph-mon[34846]: Deploying daemon mds.e on smithi092 2024-05-10T21:03:41.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:40 smithi142 ceph-mon[34846]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:41.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:40 smithi050 ceph-mon[29546]: Deploying daemon mds.e on smithi092 2024-05-10T21:03:41.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:40 smithi050 ceph-mon[29546]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:41.623 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:03:41 smithi092 systemd[1]: Started Ceph mds.e for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:41.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:42.479 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:42.770 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.770 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.770 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.770 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: Deploying daemon mds.f on smithi142 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6834/1464775005,v1:172.21.15.92:6835/1464775005] up:boot 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: fsmap 5 up:standby 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:42.771 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:42.785 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.785 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.785 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: Deploying daemon mds.f on smithi142 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6834/1464775005,v1:172.21.15.92:6835/1464775005] up:boot 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: fsmap 5 up:standby 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:42.786 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: Deploying daemon mds.f on smithi142 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6834/1464775005,v1:172.21.15.92:6835/1464775005] up:boot 2024-05-10T21:03:42.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: fsmap 5 up:standby 2024-05-10T21:03:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:42.875 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:03:43.856 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:43 smithi142 systemd[1]: Starting Ceph mds.f for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:44.135 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:43 smithi142 podman[65258]: 2024-05-10 21:03:43.852299598 +0000 UTC m=+0.113858763 container create d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T21:03:44.135 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:43 smithi142 podman[65258]: 2024-05-10 21:03:43.756384173 +0000 UTC m=+0.017943310 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:44.135 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 podman[65258]: 2024-05-10 21:03:44.096210093 +0000 UTC m=+0.357769213 container init d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-10T21:03:44.135 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 podman[65258]: 2024-05-10 21:03:44.098714377 +0000 UTC m=+0.360273497 container start d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-10T21:03:44.605 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mds[65277]: -- 172.21.15.142:0/3205353315 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582cf2cda40 con 0x5582cf315600 2024-05-10T21:03:44.605 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f[65273]: starting mds.f at 2024-05-10T21:03:44.605 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 bash[65258]: d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 2024-05-10T21:03:44.605 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:03:44 smithi142 systemd[1]: Started Ceph mds.f for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 -- 172.21.15.50:0/1865165266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 msgr2=0x7f5398114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 --2- 172.21.15.50:0/1865165266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f5398114e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5384009990 tx=0x7f5384009ca0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 -- 172.21.15.50:0/1865165266 shutdown_connections 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 --2- 172.21.15.50:0/1865165266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398115370 0x7f5398117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 --2- 172.21.15.50:0/1865165266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398075b60 0x7f5398075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 --2- 172.21.15.50:0/1865165266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f5398114e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 -- 172.21.15.50:0/1865165266 >> 172.21.15.50:0/1865165266 conn(0x7f5398100040 msgr2=0x7f5398102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 -- 172.21.15.50:0/1865165266 shutdown_connections 2024-05-10T21:03:44.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.847+0000 7f539d800700 1 -- 172.21.15.50:0/1865165266 wait complete. 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:44.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.849+0000 7f539d800700 1 Processor -- start 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.849+0000 7f539d800700 1 -- start start 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398115370 0x7f53981a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f539811aac0 con 0x7f5398115370 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f539811a940 con 0x7f5398075b60 2024-05-10T21:03:44.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f539d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f539811ac40 con 0x7f53980764b0 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5396000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5396a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5396a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48702/0 (socket says 172.21.15.50:48702) 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5396000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60042/0 (socket says 172.21.15.50:60042) 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5396a00700 1 -- 172.21.15.50:0/1624101914 learned_addr learned my addr 172.21.15.50:0/1624101914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:44.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.850+0000 7f5397400700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398115370 0x7f53981a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 -- 172.21.15.50:0/1624101914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 msgr2=0x7f53981a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 -- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398115370 msgr2=0x7f53981a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398115370 0x7f53981a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 -- 172.21.15.50:0/1624101914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53981a6320 con 0x7f53980764b0 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396a00700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f5396000700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f53840047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.851+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384026070 con 0x7f53980764b0 2024-05-10T21:03:44.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.852+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5384021410 con 0x7f53980764b0 2024-05-10T21:03:44.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.852+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53981a65b0 con 0x7f53980764b0 2024-05-10T21:03:44.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.852+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384017650 con 0x7f53980764b0 2024-05-10T21:03:44.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.852+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53981adf30 con 0x7f53980764b0 2024-05-10T21:03:44.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.853+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53980772a0 con 0x7f53980764b0 2024-05-10T21:03:44.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.856+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5384029070 con 0x7f53980764b0 2024-05-10T21:03:44.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.857+0000 7f538f400700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 0x7f537c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:44.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.857+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f53840e2b80 con 0x7f53980764b0 2024-05-10T21:03:44.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.857+0000 7f5396a00700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 0x7f537c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:44.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.858+0000 7f5396a00700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 0x7f537c0b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5388011550 tx=0x7f5388009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:44.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:44.862+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5384013070 con 0x7f53980764b0 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:44.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:44.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:44.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:45.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.148+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53981a27b0 con 0x7f53980764b0 2024-05-10T21:03:45.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.149+0000 7f538f400700 1 -- 172.21.15.50:0/1624101914 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f53840aad30 con 0x7f53980764b0 2024-05-10T21:03:45.154 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:45.154 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.152+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 msgr2=0x7f537c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.152+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 0x7f537c0b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5388011550 tx=0x7f5388009260 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.152+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 msgr2=0x7f53981a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.152+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f53840047b0 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 shutdown_connections 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f537c0b2900 0x7f537c0b4e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5398075b60 0x7f53981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5398115370 0x7f53981a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 --2- 172.21.15.50:0/1624101914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53980764b0 0x7f53981a19f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 >> 172.21.15.50:0/1624101914 conn(0x7f5398100040 msgr2=0x7f5398100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 shutdown_connections 2024-05-10T21:03:45.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:45.153+0000 7f539d800700 1 -- 172.21.15.50:0/1624101914 wait complete. 2024-05-10T21:03:45.159 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 7 2024-05-10T21:03:45.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:45 smithi142 ceph-mon[34846]: Deploying daemon mds.c on smithi142 2024-05-10T21:03:45.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:45 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6834/86448197,v1:172.21.15.142:6835/86448197] up:boot 2024-05-10T21:03:45.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:45 smithi142 ceph-mon[34846]: fsmap 6 up:standby 2024-05-10T21:03:45.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:45 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T21:03:45.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:45 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1624101914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:45.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:45 smithi050 ceph-mon[29546]: Deploying daemon mds.c on smithi142 2024-05-10T21:03:45.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:45 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6834/86448197,v1:172.21.15.142:6835/86448197] up:boot 2024-05-10T21:03:45.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:45 smithi050 ceph-mon[29546]: fsmap 6 up:standby 2024-05-10T21:03:45.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:45 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T21:03:45.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:45 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1624101914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:46.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:45 smithi092 ceph-mon[34415]: Deploying daemon mds.c on smithi142 2024-05-10T21:03:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:45 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6834/86448197,v1:172.21.15.142:6835/86448197] up:boot 2024-05-10T21:03:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:45 smithi092 ceph-mon[34415]: fsmap 6 up:standby 2024-05-10T21:03:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:45 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T21:03:46.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:45 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1624101914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:46.633 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:46 smithi142 systemd[1]: Starting Ceph mds.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:46.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:46 smithi142 ceph-mon[34846]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:46.885 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:46 smithi142 podman[65569]: 2024-05-10 21:03:46.618742101 +0000 UTC m=+0.017366925 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:46.885 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:46 smithi142 podman[65569]: 2024-05-10 21:03:46.726962491 +0000 UTC m=+0.125587348 container create 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:03:46.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:46 smithi050 ceph-mon[29546]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:47.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:46 smithi092 ceph-mon[34415]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:47.226 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:46 smithi142 podman[65569]: 2024-05-10 21:03:46.970990352 +0000 UTC m=+0.369615171 container init 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-10T21:03:47.227 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:46 smithi142 podman[65569]: 2024-05-10 21:03:46.973828342 +0000 UTC m=+0.372453153 container start 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:03:47.227 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:47 smithi142 ceph-mds[65589]: -- 172.21.15.142:0/675868375 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55692f467a40 con 0x55692f4afa80 2024-05-10T21:03:47.227 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:47 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: starting mds.c at 2024-05-10T21:03:47.227 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:47 smithi142 bash[65569]: 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 2024-05-10T21:03:47.634 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:03:47 smithi142 systemd[1]: Started Ceph mds.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:47.979 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:48.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: Deploying daemon mds.h on smithi092 2024-05-10T21:03:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6836/924361476,v1:172.21.15.142:6837/924361476] up:boot 2024-05-10T21:03:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: fsmap 7 up:standby 2024-05-10T21:03:48.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:03:48.450 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:48.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: Deploying daemon mds.h on smithi092 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6836/924361476,v1:172.21.15.142:6837/924361476] up:boot 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: fsmap 7 up:standby 2024-05-10T21:03:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:03:48.634 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.634 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: Deploying daemon mds.h on smithi092 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6836/924361476,v1:172.21.15.142:6837/924361476] up:boot 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: fsmap 7 up:standby 2024-05-10T21:03:48.635 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:03:49.624 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:49 smithi092 systemd[1]: Starting Ceph mds.h for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:50.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:49 smithi092 podman[64486]: 2024-05-10 21:03:49.592299622 +0000 UTC m=+0.017996548 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:50.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:49 smithi092 podman[64486]: 2024-05-10 21:03:49.736372328 +0000 UTC m=+0.162069254 container create 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-10T21:03:50.001 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:49 smithi092 podman[64486]: 2024-05-10 21:03:49.997520004 +0000 UTC m=+0.423216938 container init 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T21:03:50.373 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:50 smithi092 podman[64486]: 2024-05-10 21:03:50.000796587 +0000 UTC m=+0.426493514 container start 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-10T21:03:50.373 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mds[64505]: -- 172.21.15.92:0/3561342759 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b59db5a40 con 0x556b59dfd600 2024-05-10T21:03:50.374 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: starting mds.h at 2024-05-10T21:03:50.374 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:50 smithi092 bash[64486]: 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 2024-05-10T21:03:50.374 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:03:50 smithi092 systemd[1]: Started Ceph mds.h for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 -- 172.21.15.50:0/1500869881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360108d30 msgr2=0x7f2360109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 --2- 172.21.15.50:0/1500869881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360108d30 0x7f2360109140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f23480099c0 tx=0x7f2348009cd0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 -- 172.21.15.50:0/1500869881 shutdown_connections 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 --2- 172.21.15.50:0/1500869881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360104400 0x7f2360104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 --2- 172.21.15.50:0/1500869881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f2360110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 --2- 172.21.15.50:0/1500869881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360108d30 0x7f2360109140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 -- 172.21.15.50:0/1500869881 >> 172.21.15.50:0/1500869881 conn(0x7f2360079d10 msgr2=0x7f236007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.660+0000 7f2365000700 1 -- 172.21.15.50:0/1500869881 shutdown_connections 2024-05-10T21:03:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.661+0000 7f2365000700 1 -- 172.21.15.50:0/1500869881 wait complete. 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:50.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:50 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:50.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.662+0000 7f2365000700 1 Processor -- start 2024-05-10T21:03:50.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.662+0000 7f2365000700 1 -- start start 2024-05-10T21:03:50.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 0x7f23601a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:50.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 0x7f23601a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:50.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f236019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2360114290 con 0x7f2360104400 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2360114110 con 0x7f2360108d30 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2365000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2360114410 con 0x7f2360109680 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f235f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 0x7f23601a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f235fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f236019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f2356a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 0x7f23601a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f235fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f236019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60054/0 (socket says 172.21.15.50:60054) 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.663+0000 7f235fe00700 1 -- 172.21.15.50:0/3772613041 learned_addr learned my addr 172.21.15.50:0/3772613041 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 -- 172.21.15.50:0/3772613041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 msgr2=0x7f236019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f236019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 -- 172.21.15.50:0/3772613041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 msgr2=0x7f23601a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 0x7f23601a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 -- 172.21.15.50:0/3772613041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236019b260 con 0x7f2360104400 2024-05-10T21:03:50.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f2356a00700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 0x7f23601a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:50.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235f400700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 0x7f23601a09a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f235000e9f0 tx=0x7f235000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:50.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000c9c0 con 0x7f2360104400 2024-05-10T21:03:50.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235000cb60 con 0x7f2360104400 2024-05-10T21:03:50.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350018680 con 0x7f2360104400 2024-05-10T21:03:50.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.664+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236019b550 con 0x7f2360104400 2024-05-10T21:03:50.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.665+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23601a7750 con 0x7f2360104400 2024-05-10T21:03:50.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.667+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23601058b0 con 0x7f2360104400 2024-05-10T21:03:50.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.669+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2350010430 con 0x7f2360104400 2024-05-10T21:03:50.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.671+0000 7f235cc00700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 0x7f23440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:50.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.671+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f23500e2210 con 0x7f2360104400 2024-05-10T21:03:50.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.671+0000 7f2356a00700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 0x7f23440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:50.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.672+0000 7f2356a00700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 0x7f23440b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2348009990 tx=0x7f234801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:50.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.674+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23500aa330 con 0x7f2360104400 2024-05-10T21:03:50.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:50.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:50.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:50.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:50 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:50 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:50.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.966+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f236002e5f0 con 0x7f2360104400 2024-05-10T21:03:50.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.967+0000 7f235cc00700 1 -- 172.21.15.50:0/3772613041 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7f23500a9c30 con 0x7f2360104400 2024-05-10T21:03:50.972 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:50.973 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-10T21:03:50.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 msgr2=0x7f23440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:50.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 0x7f23440b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2348009990 tx=0x7f234801a040 comp rx=0 tx=0).stop 2024-05-10T21:03:50.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 msgr2=0x7f23601a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 0x7f23601a09a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f235000e9f0 tx=0x7f235000ed00 comp rx=0 tx=0).stop 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 shutdown_connections 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23440b2900 0x7f23440b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2360109680 0x7f236019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2360104400 0x7f23601a09a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 --2- 172.21.15.50:0/3772613041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2360108d30 0x7f23601a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 >> 172.21.15.50:0/3772613041 conn(0x7f2360079d10 msgr2=0x7f2360101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 shutdown_connections 2024-05-10T21:03:50.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:50.971+0000 7f2365000700 1 -- 172.21.15.50:0/3772613041 wait complete. 2024-05-10T21:03:50.977 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-05-10T21:03:51.897 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:51 smithi142 ceph-mon[34846]: Deploying daemon mds.i on smithi142 2024-05-10T21:03:51.897 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:51 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6836/1752993045,v1:172.21.15.92:6837/1752993045] up:boot 2024-05-10T21:03:51.897 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:51 smithi142 ceph-mon[34846]: fsmap 8 up:standby 2024-05-10T21:03:51.897 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:51 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:03:51.897 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:51 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3772613041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:51.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:51 smithi050 ceph-mon[29546]: Deploying daemon mds.i on smithi142 2024-05-10T21:03:51.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:51 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6836/1752993045,v1:172.21.15.92:6837/1752993045] up:boot 2024-05-10T21:03:51.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:51 smithi050 ceph-mon[29546]: fsmap 8 up:standby 2024-05-10T21:03:51.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:51 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:03:51.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:51 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3772613041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:52.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:51 smithi092 ceph-mon[34415]: Deploying daemon mds.i on smithi142 2024-05-10T21:03:52.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:51 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6836/1752993045,v1:172.21.15.92:6837/1752993045] up:boot 2024-05-10T21:03:52.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:51 smithi092 ceph-mon[34415]: fsmap 8 up:standby 2024-05-10T21:03:52.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:51 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:03:52.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:51 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3772613041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:52.634 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 podman[65877]: 2024-05-10 21:03:52.530573939 +0000 UTC m=+0.137177003 container create 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-10T21:03:52.903 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:52 smithi142 ceph-mon[34846]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:52.904 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 podman[65877]: 2024-05-10 21:03:52.841537538 +0000 UTC m=+0.448140527 container init 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-10T21:03:52.904 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 podman[65877]: 2024-05-10 21:03:52.844303716 +0000 UTC m=+0.450906705 container start 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux ) 2024-05-10T21:03:52.904 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 ceph-mds[65896]: -- 172.21.15.142:0/1063581526 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55988f217a40 con 0x55988f25fa80 2024-05-10T21:03:52.904 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: starting mds.i at 2024-05-10T21:03:52.904 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 bash[65877]: 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e 2024-05-10T21:03:52.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:52 smithi050 ceph-mon[29546]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:53.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:52 smithi092 ceph-mon[34415]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:53.384 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:03:52 smithi142 systemd[1]: Started Ceph mds.i for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:53.828 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:54.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: Deploying daemon mds.g on smithi050 2024-05-10T21:03:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:boot 2024-05-10T21:03:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: fsmap 9 up:standby 2024-05-10T21:03:54.174 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: Deploying daemon mds.g on smithi050 2024-05-10T21:03:54.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:54.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:boot 2024-05-10T21:03:54.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: fsmap 9 up:standby 2024-05-10T21:03:54.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:54.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: Deploying daemon mds.g on smithi050 2024-05-10T21:03:54.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:54.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:boot 2024-05-10T21:03:54.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: fsmap 9 up:standby 2024-05-10T21:03:54.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:03:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:03:56.173 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:55 smithi050 systemd[1]: Starting Ceph mds.g for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:03:56.813 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:56 smithi050 ceph-mon[29546]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:56.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:56 smithi050 podman[69000]: 2024-05-10 21:03:56.65012579 +0000 UTC m=+0.038841240 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:03:56.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:56 smithi092 ceph-mon[34415]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:56.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:56 smithi142 ceph-mon[34846]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:57.097 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:56 smithi050 podman[69000]: 2024-05-10 21:03:56.808935467 +0000 UTC m=+0.197650948 container create 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-10T21:03:57.097 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 podman[69000]: 2024-05-10 21:03:57.090597746 +0000 UTC m=+0.479313274 container init 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-10T21:03:57.385 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 podman[69000]: 2024-05-10 21:03:57.105036672 +0000 UTC m=+0.493752163 container start 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-10T21:03:57.386 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 ceph-mds[69035]: -- 172.21.15.50:0/507540239 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f07c205a40 con 0x55f07c24d600 2024-05-10T21:03:57.386 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g[69029]: starting mds.g at 2024-05-10T21:03:57.672 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 bash[69000]: 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 2024-05-10T21:03:57.673 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:03:57 smithi050 systemd[1]: Started Ceph mds.g for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:03:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 -- 172.21.15.50:0/647696901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28110ae0 msgr2=0x7efd28112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 --2- 172.21.15.50:0/647696901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28110ae0 0x7efd28112f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd2000b0a0 tx=0x7efd2000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 -- 172.21.15.50:0/647696901 shutdown_connections 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 --2- 172.21.15.50:0/647696901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28102f30 0x7efd28103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 --2- 172.21.15.50:0/647696901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28110ae0 0x7efd28112f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 --2- 172.21.15.50:0/647696901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28103880 0x7efd281105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 -- 172.21.15.50:0/647696901 >> 172.21.15.50:0/647696901 conn(0x7efd28078ac0 msgr2=0x7efd28078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.750+0000 7efd2d600700 1 -- 172.21.15.50:0/647696901 shutdown_connections 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.751+0000 7efd2d600700 1 -- 172.21.15.50:0/647696901 wait complete. 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 Processor -- start 2024-05-10T21:03:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 -- start start 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28102f30 0x7efd2819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 0x7efd281a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd28116500 con 0x7efd28102f30 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd28116380 con 0x7efd28103880 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.752+0000 7efd2d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd28116680 con 0x7efd28110ae0 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59252/0 (socket says 172.21.15.50:59252) 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 -- 172.21.15.50:0/2787462926 learned_addr learned my addr 172.21.15.50:0/2787462926 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:03:57.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27e00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28102f30 0x7efd2819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd2cc00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 0x7efd281a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 -- 172.21.15.50:0/2787462926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 msgr2=0x7efd281a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 0x7efd281a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 -- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28102f30 msgr2=0x7efd2819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28102f30 0x7efd2819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd27400700 1 -- 172.21.15.50:0/2787462926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281a1ce0 con 0x7efd28103880 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.753+0000 7efd2cc00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 0x7efd281a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd27400700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd1400ea10 tx=0x7efd1400ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1400cbd0 con 0x7efd28103880 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd1400cd70 con 0x7efd28103880 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14018680 con 0x7efd28103880 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd281a1fd0 con 0x7efd28103880 2024-05-10T21:03:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.754+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd281a98f0 con 0x7efd28103880 2024-05-10T21:03:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.755+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd280512c0 con 0x7efd28103880 2024-05-10T21:03:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.758+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efd1401f020 con 0x7efd28103880 2024-05-10T21:03:57.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.759+0000 7efd24c00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 0x7efd0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:03:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.759+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7efd140e1c00 con 0x7efd28103880 2024-05-10T21:03:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.760+0000 7efd27e00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 0x7efd0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:03:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.760+0000 7efd27e00700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 0x7efd0c0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd18009c60 tx=0x7efd18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:03:57.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:57.764+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd140a9d20 con 0x7efd28103880 2024-05-10T21:03:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.053+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd2802e5f0 con 0x7efd28103880 2024-05-10T21:03:58.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.054+0000 7efd24c00700 1 -- 172.21.15.50:0/2787462926 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7efd140a9620 con 0x7efd28103880 2024-05-10T21:03:58.059 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:03:58.059 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T21:03:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 msgr2=0x7efd0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 0x7efd0c0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd18009c60 tx=0x7efd18009260 comp rx=0 tx=0).stop 2024-05-10T21:03:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 msgr2=0x7efd2819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:03:58.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd1400ea10 tx=0x7efd1400ed20 comp rx=0 tx=0).stop 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 shutdown_connections 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd0c0b2900 0x7efd0c0b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd28110ae0 0x7efd281a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd28102f30 0x7efd2819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 --2- 172.21.15.50:0/2787462926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd28103880 0x7efd2819d3b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.057+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 >> 172.21.15.50:0/2787462926 conn(0x7efd28078ac0 msgr2=0x7efd28111d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.058+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 shutdown_connections 2024-05-10T21:03:58.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:03:58.058+0000 7efd2d600700 1 -- 172.21.15.50:0/2787462926 wait complete. 2024-05-10T21:03:58.064 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 11 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6838/82705098,v1:172.21.15.50:6839/82705098] up:boot 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: fsmap 10 up:standby 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:58.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T21:03:58.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: Deploying daemon mds.a on smithi050 2024-05-10T21:03:58.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:03:58 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2787462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:58.825 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.825 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:58.825 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6838/82705098,v1:172.21.15.50:6839/82705098] up:boot 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: fsmap 10 up:standby 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:58.826 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T21:03:58.827 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: Deploying daemon mds.a on smithi050 2024-05-10T21:03:58.827 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:03:58 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2787462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:03:58.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6838/82705098,v1:172.21.15.50:6839/82705098] up:boot 2024-05-10T21:03:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: fsmap 10 up:standby 2024-05-10T21:03:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:03:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T21:03:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: Deploying daemon mds.a on smithi050 2024-05-10T21:03:58.885 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:03:58 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2787462926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:00.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:00 smithi092 ceph-mon[34415]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:00.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:00 smithi142 ceph-mon[34846]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:00.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:00 smithi050 ceph-mon[29546]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:00.931 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:04:01.408 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 systemd[1]: Starting Ceph mds.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:04:01.673 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 podman[69389]: 2024-05-10 21:04:01.308401707 +0000 UTC m=+0.035732359 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:04:01.674 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 podman[69389]: 2024-05-10 21:04:01.428705544 +0000 UTC m=+0.156036228 container create 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:04:01.832 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 podman[69389]: 2024-05-10 21:04:01.695552238 +0000 UTC m=+0.422882925 container init 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507) 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 podman[69389]: 2024-05-10 21:04:01.706635742 +0000 UTC m=+0.433966401 container start 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 bash[69389]: 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 systemd[1]: Started Ceph mds.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 ceph-mds[69436]: -- 172.21.15.50:0/3383756736 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff42417a40 con 0x55ff4245f180 2024-05-10T21:04:01.987 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:04:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: starting mds.a at 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:04:02.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:04:02.884 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:02.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:02.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:02.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T21:04:02.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T21:04:02.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:03.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:03 smithi092 ceph-mon[34415]: Deploying daemon mds.b on smithi092 2024-05-10T21:04:03.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:03 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:boot 2024-05-10T21:04:03.754 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:03 smithi092 ceph-mon[34415]: fsmap 11 up:standby 2024-05-10T21:04:03.755 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:04:03.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:03 smithi050 ceph-mon[29546]: Deploying daemon mds.b on smithi092 2024-05-10T21:04:03.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:03 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:boot 2024-05-10T21:04:03.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:03 smithi050 ceph-mon[29546]: fsmap 11 up:standby 2024-05-10T21:04:03.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:04:04.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 -- 172.21.15.50:0/3928562190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 msgr2=0x7f8564104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 --2- 172.21.15.50:0/3928562190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f8564104810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8558009990 tx=0x7f8558009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 -- 172.21.15.50:0/3928562190 shutdown_connections 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 --2- 172.21.15.50:0/3928562190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f8564104810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 --2- 172.21.15.50:0/3928562190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f8564110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 --2- 172.21.15.50:0/3928562190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f8564109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 -- 172.21.15.50:0/3928562190 >> 172.21.15.50:0/3928562190 conn(0x7f8564079d10 msgr2=0x7f856407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 -- 172.21.15.50:0/3928562190 shutdown_connections 2024-05-10T21:04:04.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.026+0000 7f856b000700 1 -- 172.21.15.50:0/3928562190 wait complete. 2024-05-10T21:04:04.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.027+0000 7f856b000700 1 Processor -- start 2024-05-10T21:04:04.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.027+0000 7f856b000700 1 -- start start 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f85641a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f856419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8564114320 con 0x7f8564108d30 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85641141a0 con 0x7f8564104400 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f856b000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85641144a0 con 0x7f8564109680 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f8569c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f8569c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54734/0 (socket says 172.21.15.50:54734) 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f8569200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f85641a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.028+0000 7f8569c00700 1 -- 172.21.15.50:0/2156278688 learned_addr learned my addr 172.21.15.50:0/2156278688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:04.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f856a600700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f856419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 -- 172.21.15.50:0/2156278688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 msgr2=0x7f856419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f856419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 -- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 msgr2=0x7f85641a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f85641a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 -- 172.21.15.50:0/2156278688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856419b270 con 0x7f8564104400 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569200700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f85641a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f856a600700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f856419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.029+0000 7f8569c00700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c00 tx=0x7f85580057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.030+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558027070 con 0x7f8564104400 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.030+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855800b610 con 0x7f8564104400 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.030+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580173f0 con 0x7f8564104400 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.030+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856419b500 con 0x7f8564104400 2024-05-10T21:04:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.030+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856419b990 con 0x7f8564104400 2024-05-10T21:04:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.031+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85641058b0 con 0x7f8564104400 2024-05-10T21:04:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.034+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8558009d10 con 0x7f8564104400 2024-05-10T21:04:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.035+0000 7f8562000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 0x7f854c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.035+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f85580e2ff0 con 0x7f8564104400 2024-05-10T21:04:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.036+0000 7f8569200700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 0x7f854c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.036+0000 7f8569200700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 0x7f854c0b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f856419c270 tx=0x7f8550012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.037+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8558013070 con 0x7f8564104400 2024-05-10T21:04:04.123 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:03 smithi092 systemd[1]: Starting Ceph mds.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:04:04.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:03 smithi142 ceph-mon[34846]: Deploying daemon mds.b on smithi092 2024-05-10T21:04:04.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:03 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:boot 2024-05-10T21:04:04.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:03 smithi142 ceph-mon[34846]: fsmap 11 up:standby 2024-05-10T21:04:04.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:04:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.325+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f856419bfe0 con 0x7f8564104400 2024-05-10T21:04:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.326+0000 7f8562000700 1 -- 172.21.15.50:0/2156278688 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7f85580ab200 con 0x7f8564104400 2024-05-10T21:04:04.331 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:04:04.331 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-10T21:04:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 msgr2=0x7f854c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 0x7f854c0b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f856419c270 tx=0x7f8550012040 comp rx=0 tx=0).stop 2024-05-10T21:04:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 msgr2=0x7f85641a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c00 tx=0x7f85580057c0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 shutdown_connections 2024-05-10T21:04:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f854c0b2900 0x7f854c0b4e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8564104400 0x7f85641a09b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8564109680 0x7f856419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 --2- 172.21.15.50:0/2156278688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564108d30 0x7f85641a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.330+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 >> 172.21.15.50:0/2156278688 conn(0x7f8564079d10 msgr2=0x7f8564102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.331+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 shutdown_connections 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:04.331+0000 7f856b000700 1 -- 172.21.15.50:0/2156278688 wait complete. 2024-05-10T21:04:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 12 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 podman[64793]: 2024-05-10 21:04:04.138730047 +0000 UTC m=+0.098680686 container create f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 podman[64793]: 2024-05-10 21:04:04.058128169 +0000 UTC m=+0.018078794 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 podman[64793]: 2024-05-10 21:04:04.324631781 +0000 UTC m=+0.284582379 container init f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 podman[64793]: 2024-05-10 21:04:04.327120908 +0000 UTC m=+0.287071511 container start f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mds[64813]: -- 172.21.15.92:0/3851840504 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557579ad3a40 con 0x557579b1ba80 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: starting mds.b at 2024-05-10T21:04:04.412 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 bash[64793]: f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d 2024-05-10T21:04:04.873 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:04 smithi092 systemd[1]: Started Ceph mds.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:04:04.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:04.873 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2156278688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.874 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:04 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:04.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2156278688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:04.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:04.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:04 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2156278688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:04 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:05 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:boot 2024-05-10T21:04:05.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:05 smithi092 ceph-mon[34415]: fsmap 12 up:standby 2024-05-10T21:04:05.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:04:05.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:05.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:05.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:05 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:boot 2024-05-10T21:04:05.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:05 smithi050 ceph-mon[29546]: fsmap 12 up:standby 2024-05-10T21:04:05.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:04:05.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:05.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:06.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:05 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:boot 2024-05-10T21:04:06.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:05 smithi142 ceph-mon[34846]: fsmap 12 up:standby 2024-05-10T21:04:06.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:04:06.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:06.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:06.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:06 smithi050 ceph-mon[29546]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:06.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:06.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:06.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:06.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:06 smithi092 ceph-mon[34415]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:07.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:06 smithi142 ceph-mon[34846]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:07.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:04:07.686 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:09.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:09.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.124 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:09.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:09.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:09.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 -- 172.21.15.50:0/1550712144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 msgr2=0x7f21fc117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 --2- 172.21.15.50:0/1550712144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc117810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b0a0 tx=0x7f21f400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 -- 172.21.15.50:0/1550712144 shutdown_connections 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 --2- 172.21.15.50:0/1550712144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc117810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 --2- 172.21.15.50:0/1550712144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc110970 0x7f21fc110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 --2- 172.21.15.50:0/1550712144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc10a990 0x7f21fc10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 -- 172.21.15.50:0/1550712144 >> 172.21.15.50:0/1550712144 conn(0x7f21fc077c80 msgr2=0x7f21fc0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:09.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 -- 172.21.15.50:0/1550712144 shutdown_connections 2024-05-10T21:04:09.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.927+0000 7f2202a00700 1 -- 172.21.15.50:0/1550712144 wait complete. 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.928+0000 7f2202a00700 1 Processor -- start 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 -- start start 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc10a990 0x7f21fc1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc11aae0 con 0x7f21fc10a990 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc11a960 con 0x7f21fc110970 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.929+0000 7f2202a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc11ac60 con 0x7f21fc10b2e0 2024-05-10T21:04:09.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:09.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f21fbe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc10a990 0x7f21fc1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:09.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54750/0 (socket says 172.21.15.50:54750) 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 -- 172.21.15.50:0/1476738871 learned_addr learned my addr 172.21.15.50:0/1476738871 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f21fb400700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 -- 172.21.15.50:0/1476738871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 msgr2=0x7f21fc1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 -- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc10a990 msgr2=0x7f21fc1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc10a990 0x7f21fc1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.930+0000 7f2200a00700 1 -- 172.21.15.50:0/1476738871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc1a62c0 con 0x7f21fc110970 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f2200a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b070 tx=0x7f21f4004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4021070 con 0x7f21fc110970 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f4007e30 con 0x7f21fc110970 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f40046d0 con 0x7f21fc110970 2024-05-10T21:04:09.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1a6550 con 0x7f21fc110970 2024-05-10T21:04:09.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.931+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a6a40 con 0x7f21fc110970 2024-05-10T21:04:09.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.932+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0512c0 con 0x7f21fc110970 2024-05-10T21:04:09.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.935+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f21f4007980 con 0x7f21fc110970 2024-05-10T21:04:09.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.936+0000 7f21f8c00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 0x7f21e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:09.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.936+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f21f400f020 con 0x7f21fc110970 2024-05-10T21:04:09.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.936+0000 7f21fbe00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 0x7f21e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:09.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.937+0000 7f21fbe00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 0x7f21e00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005950 tx=0x7f21ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:09.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:09.940+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21f402a020 con 0x7f21fc110970 2024-05-10T21:04:10.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.233+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21fc02e5f0 con 0x7f21fc110970 2024-05-10T21:04:10.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.234+0000 7f21f8c00700 1 -- 172.21.15.50:0/1476738871 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7f21f40cbe20 con 0x7f21fc110970 2024-05-10T21:04:10.239 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:04:10.239 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.238+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 msgr2=0x7f21e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.238+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 0x7f21e00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005950 tx=0x7f21ec00b410 comp rx=0 tx=0).stop 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.238+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 msgr2=0x7f21fc1a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.238+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b070 tx=0x7f21f4004cf0 comp rx=0 tx=0).stop 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.238+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 shutdown_connections 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f21e00b2900 0x7f21e00b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21fc110970 0x7f21fc1a5c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21fc10a990 0x7f21fc1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:10.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 --2- 172.21.15.50:0/1476738871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21fc10b2e0 0x7f21fc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:10.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 >> 172.21.15.50:0/1476738871 conn(0x7f21fc077c80 msgr2=0x7f21fc07b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:10.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 shutdown_connections 2024-05-10T21:04:10.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:10.239+0000 7f2202a00700 1 -- 172.21.15.50:0/1476738871 wait complete. 2024-05-10T21:04:10.244 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 13 2024-05-10T21:04:10.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:10.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:10.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:10.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:10.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:10.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:10 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1476738871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:10 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1476738871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:11.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:10 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1476738871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:04:11.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-10T21:04:11.889 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:13.115 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:12 smithi050 ceph-mon[29546]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:13.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:12 smithi092 ceph-mon[34415]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:13.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:12 smithi142 ceph-mon[34846]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:13.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.988+0000 7ffaa6200700 1 -- 172.21.15.50:0/3771003780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa010a990 msgr2=0x7ffaa010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:13.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.988+0000 7ffaa6200700 1 --2- 172.21.15.50:0/3771003780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa010a990 0x7ffaa010ada0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009990 tx=0x7ffa8c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 -- 172.21.15.50:0/3771003780 shutdown_connections 2024-05-10T21:04:13.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 --2- 172.21.15.50:0/3771003780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010b2e0 0x7ffaa0117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 --2- 172.21.15.50:0/3771003780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa0110970 0x7ffaa0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 --2- 172.21.15.50:0/3771003780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa010a990 0x7ffaa010ada0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 -- 172.21.15.50:0/3771003780 >> 172.21.15.50:0/3771003780 conn(0x7ffaa0077c80 msgr2=0x7ffaa00780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:13.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 -- 172.21.15.50:0/3771003780 shutdown_connections 2024-05-10T21:04:13.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.989+0000 7ffaa6200700 1 -- 172.21.15.50:0/3771003780 wait complete. 2024-05-10T21:04:13.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:13.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:13.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:13.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:13.995 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.990+0000 7ffaa6200700 1 Processor -- start 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.990+0000 7ffaa6200700 1 -- start start 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa010b2e0 0x7ffaa01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa0110970 0x7ffaa01a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa011aa90 con 0x7ffaa0110970 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa011a910 con 0x7ffaa010a990 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffaa6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa011ac10 con 0x7ffaa010b2e0 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffa9f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffa9f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48268/0 (socket says 172.21.15.50:48268) 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffa9f400700 1 -- 172.21.15.50:0/2353433876 learned_addr learned my addr 172.21.15.50:0/2353433876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:13.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffa9fe00700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa0110970 0x7ffaa01a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.991+0000 7ffa9ea00700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa010b2e0 0x7ffaa01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 -- 172.21.15.50:0/2353433876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa010b2e0 msgr2=0x7ffaa01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa010b2e0 0x7ffaa01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 -- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa0110970 msgr2=0x7ffaa01a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa0110970 0x7ffaa01a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:13.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 -- 172.21.15.50:0/2353433876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa01a6390 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa9f400700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000e970 tx=0x7ffa9000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90004a30 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9000cc90 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90005610 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.992+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa01a6680 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.993+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa01adfa0 con 0x7ffaa010a990 2024-05-10T21:04:13.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.994+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa00512c0 con 0x7ffaa010a990 2024-05-10T21:04:14.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.996+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffa9000c7e0 con 0x7ffaa010a990 2024-05-10T21:04:14.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.997+0000 7ffa97e00700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:14.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.998+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ffa900e1eb0 con 0x7ffaa010a990 2024-05-10T21:04:14.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.998+0000 7ffa9ea00700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:14.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:13.998+0000 7ffa9ea00700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 0x7ffa840b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:14.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.002+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa900a9fd0 con 0x7ffaa010a990 2024-05-10T21:04:14.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:14.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:14.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:04:14.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:04:14.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:04:14.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.239+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffaa01a2820 con 0x7ffaa010a990 2024-05-10T21:04:14.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.240+0000 7ffa97e00700 1 -- 172.21.15.50:0/2353433876 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7ffa900a98d0 con 0x7ffaa010a990 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:04:14.245 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 msgr2=0x7ffa840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 0x7ffa840b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c005960 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 msgr2=0x7ffaa01a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000e970 tx=0x7ffa9000ec80 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 shutdown_connections 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa840b2900 0x7ffa840b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa010a990 0x7ffaa01a14c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffaa0110970 0x7ffaa01a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 --2- 172.21.15.50:0/2353433876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaa010b2e0 0x7ffaa01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.244+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 >> 172.21.15.50:0/2353433876 conn(0x7ffaa0077c80 msgr2=0x7ffaa007b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.245+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 shutdown_connections 2024-05-10T21:04:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:14.245+0000 7ffaa6200700 1 -- 172.21.15.50:0/2353433876 wait complete. 2024-05-10T21:04:14.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:14 smithi050 ceph-mon[29546]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:14.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:14 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2353433876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T21:04:15.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:14 smithi092 ceph-mon[34415]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:15.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:14 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2353433876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T21:04:15.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:14 smithi142 ceph-mon[34846]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:15.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:14 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2353433876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T21:04:15.158 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T21:04:15.158 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T21:04:15.627 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:17.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:16 smithi092 ceph-mon[34415]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:17.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:16 smithi142 ceph-mon[34846]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:17.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:16 smithi050 ceph-mon[29546]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.782+0000 7f7693800700 1 -- 172.21.15.50:0/68953863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c108f20 msgr2=0x7f768c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.782+0000 7f7693800700 1 --2- 172.21.15.50:0/68953863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c108f20 0x7f768c109330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7678009990 tx=0x7f7678009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 -- 172.21.15.50:0/68953863 shutdown_connections 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 --2- 172.21.15.50:0/68953863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c109870 0x7f768c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 --2- 172.21.15.50:0/68953863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c1045f0 0x7f768c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 --2- 172.21.15.50:0/68953863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c108f20 0x7f768c109330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 -- 172.21.15.50:0/68953863 >> 172.21.15.50:0/68953863 conn(0x7f768c100040 msgr2=0x7f768c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:17.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 -- 172.21.15.50:0/68953863 shutdown_connections 2024-05-10T21:04:17.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.783+0000 7f7693800700 1 -- 172.21.15.50:0/68953863 wait complete. 2024-05-10T21:04:17.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.784+0000 7f7693800700 1 Processor -- start 2024-05-10T21:04:17.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.784+0000 7f7693800700 1 -- start start 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c1045f0 0x7f768c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c109870 0x7f768c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1144d0 con 0x7f768c108f20 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f768c114350 con 0x7f768c1045f0 2024-05-10T21:04:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7693800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f768c114650 con 0x7f768c109870 2024-05-10T21:04:17.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f768be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:17.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f768be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55980/0 (socket says 172.21.15.50:55980) 2024-05-10T21:04:17.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f768be00700 1 -- 172.21.15.50:0/2245919996 learned_addr learned my addr 172.21.15.50:0/2245919996 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7691800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c109870 0x7f768c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.785+0000 7f7690e00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c1045f0 0x7f768c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 -- 172.21.15.50:0/2245919996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c109870 msgr2=0x7f768c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c109870 0x7f768c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 -- 172.21.15.50:0/2245919996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c1045f0 msgr2=0x7f768c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c1045f0 0x7f768c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 -- 172.21.15.50:0/2245919996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c1a92c0 con 0x7f768c108f20 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f768be00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7678000c00 tx=0x7f7678009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.786+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678026070 con 0x7f768c108f20 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.787+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767800b610 con 0x7f768c108f20 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.787+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c1a9550 con 0x7f768c108f20 2024-05-10T21:04:17.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.787+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678021e40 con 0x7f768c108f20 2024-05-10T21:04:17.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.787+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c1a99a0 con 0x7f768c108f20 2024-05-10T21:04:17.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.788+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c0512c0 con 0x7f768c108f20 2024-05-10T21:04:17.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.790+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7678029070 con 0x7f768c108f20 2024-05-10T21:04:17.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.792+0000 7f7689600700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 0x7f76740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:17.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.792+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f76780e2e80 con 0x7f768c108f20 2024-05-10T21:04:17.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.792+0000 7f7690e00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 0x7f76740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:17.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.793+0000 7f7690e00700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 0x7f76740b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7680009c60 tx=0x7f7680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:17.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:17.795+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7678013070 con 0x7f768c108f20 2024-05-10T21:04:18.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.031+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f768c02e5f0 con 0x7f768c108f20 2024-05-10T21:04:18.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.836+0000 7f7689600700 1 -- 172.21.15.50:0/2245919996 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f76780ab030 con 0x7f768c108f20 2024-05-10T21:04:18.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.841+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 msgr2=0x7f76740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.841+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 0x7f76740b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7680009c60 tx=0x7f7680009260 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.841+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 msgr2=0x7f768c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.841+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7678000c00 tx=0x7f7678009580 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 shutdown_connections 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2880 0x7f76740b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c1045f0 0x7f768c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c109870 0x7f768c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 --2- 172.21.15.50:0/2245919996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c108f20 0x7f768c076000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:18.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.842+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 >> 172.21.15.50:0/2245919996 conn(0x7f768c100040 msgr2=0x7f768c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:18.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.844+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 shutdown_connections 2024-05-10T21:04:18.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:18.844+0000 7f7693800700 1 -- 172.21.15.50:0/2245919996 wait complete. 2024-05-10T21:04:18.850 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-05-10T21:04:19.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:18 smithi092 ceph-mon[34415]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:19.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:18 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T21:04:19.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:18 smithi142 ceph-mon[34846]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:19.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:18 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T21:04:19.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:18 smithi050 ceph-mon[29546]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:19.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:18 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T21:04:19.737 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T21:04:19.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T21:04:19.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:19 smithi050 ceph-mon[29546]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T21:04:20.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T21:04:20.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:19 smithi092 ceph-mon[34415]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T21:04:20.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2245919996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T21:04:20.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:19 smithi142 ceph-mon[34846]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T21:04:20.243 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:21.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:20 smithi092 ceph-mon[34415]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:21.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:20 smithi092 ceph-mon[34415]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:21.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:20 smithi092 ceph-mon[34415]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T21:04:21.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:20 smithi142 ceph-mon[34846]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:21.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:20 smithi142 ceph-mon[34846]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:21.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:20 smithi142 ceph-mon[34846]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T21:04:21.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:20 smithi050 ceph-mon[29546]: pgmap v316: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:21.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:20 smithi050 ceph-mon[29546]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:21.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:20 smithi050 ceph-mon[29546]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T21:04:21.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:21 smithi050 ceph-mon[29546]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T21:04:22.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:21 smithi142 ceph-mon[34846]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T21:04:22.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:21 smithi092 ceph-mon[34415]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T21:04:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.587+0000 7f7475600700 1 -- 172.21.15.50:0/84353051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 msgr2=0x7f7470113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.587+0000 7f7475600700 1 --2- 172.21.15.50:0/84353051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f7470113370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f746800b0a0 tx=0x7f746800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 -- 172.21.15.50:0/84353051 shutdown_connections 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 --2- 172.21.15.50:0/84353051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f7470113370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 --2- 172.21.15.50:0/84353051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 0x7f747010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 --2- 172.21.15.50:0/84353051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f74701069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 -- 172.21.15.50:0/84353051 >> 172.21.15.50:0/84353051 conn(0x7f7470102090 msgr2=0x7f74701044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 -- 172.21.15.50:0/84353051 shutdown_connections 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.588+0000 7f7475600700 1 -- 172.21.15.50:0/84353051 wait complete. 2024-05-10T21:04:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.590+0000 7f7475600700 1 Processor -- start 2024-05-10T21:04:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.590+0000 7f7475600700 1 -- start start 2024-05-10T21:04:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.590+0000 7f7475600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.590+0000 7f7475600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f747019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f7475600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 0x7f74701a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f7475600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7470116750 con 0x7f747010c5d0 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f7475600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74701165d0 con 0x7f7470106ee0 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f7475600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74701168d0 con 0x7f7470106590 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50334/0 (socket says 172.21.15.50:50334) 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 -- 172.21.15.50:0/1504343561 learned_addr learned my addr 172.21.15.50:0/1504343561 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746e000700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f747019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746f400700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 0x7f74701a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:22.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 -- 172.21.15.50:0/1504343561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 msgr2=0x7f747019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:22.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f747019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 -- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 msgr2=0x7f74701a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 0x7f74701a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.591+0000 7f746ea00700 1 -- 172.21.15.50:0/1504343561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74701a1f50 con 0x7f7470106590 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f746e000700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f747019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f746ea00700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f746000e970 tx=0x7f746000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460004a30 con 0x7f7470106590 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f746000cc90 con 0x7f7470106590 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460005610 con 0x7f7470106590 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.592+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74701a2240 con 0x7f7470106590 2024-05-10T21:04:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.593+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74701a9b60 con 0x7f7470106590 2024-05-10T21:04:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.594+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74700512c0 con 0x7f7470106590 2024-05-10T21:04:22.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.597+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f746000c7e0 con 0x7f7470106590 2024-05-10T21:04:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.598+0000 7f7467400700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 0x7f74540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.598+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f74600e2240 con 0x7f7470106590 2024-05-10T21:04:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.599+0000 7f746e000700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 0x7f74540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.599+0000 7f746e000700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 0x7f74540b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7470103ba0 tx=0x7f745c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.600+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7460054df0 con 0x7f7470106590 2024-05-10T21:04:22.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.844+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f747002e5f0 con 0x7f7470106590 2024-05-10T21:04:22.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.893+0000 7f7467400700 1 -- 172.21.15.50:0/1504343561 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f74600aa1f0 con 0x7f7470106590 2024-05-10T21:04:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.900+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 msgr2=0x7f74540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.900+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 0x7f74540b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7470103ba0 tx=0x7f745c009260 comp rx=0 tx=0).stop 2024-05-10T21:04:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.900+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 msgr2=0x7f747019d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:22.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.900+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f746000e970 tx=0x7f746000ec80 comp rx=0 tx=0).stop 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 shutdown_connections 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74540b2900 0x7f74540b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f747010c5d0 0x7f74701a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7470106590 0x7f747019d080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 --2- 172.21.15.50:0/1504343561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7470106ee0 0x7f747019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.902+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 >> 172.21.15.50:0/1504343561 conn(0x7f7470102090 msgr2=0x7f7470104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:22.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.904+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 shutdown_connections 2024-05-10T21:04:22.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:22.904+0000 7f7475600700 1 -- 172.21.15.50:0/1504343561 wait complete. 2024-05-10T21:04:22.910 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-05-10T21:04:23.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:22 smithi142 ceph-mon[34846]: pgmap v319: 33 pgs: 12 active+clean, 2 creating+activating, 3 creating+peering, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:23.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:22 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1504343561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:22 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:22 smithi050 ceph-mon[29546]: pgmap v319: 33 pgs: 12 active+clean, 2 creating+activating, 3 creating+peering, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:23.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:22 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1504343561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:22 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:22 smithi092 ceph-mon[34415]: pgmap v319: 33 pgs: 12 active+clean, 2 creating+activating, 3 creating+peering, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:23.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:22 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1504343561' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:22 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T21:04:23.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T21:04:24.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:23 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T21:04:24.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:23 smithi050 ceph-mon[29546]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T21:04:24.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:23 smithi050 ceph-mon[29546]: pgmap v321: 97 pgs: 25 active+clean, 2 creating+activating, 3 creating+peering, 67 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:24.252 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:24.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:23 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T21:04:24.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:23 smithi092 ceph-mon[34415]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T21:04:24.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:23 smithi092 ceph-mon[34415]: pgmap v321: 97 pgs: 25 active+clean, 2 creating+activating, 3 creating+peering, 67 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:24.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:23 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T21:04:24.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:23 smithi142 ceph-mon[34846]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T21:04:24.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:23 smithi142 ceph-mon[34846]: pgmap v321: 97 pgs: 25 active+clean, 2 creating+activating, 3 creating+peering, 67 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:25.123 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:24 smithi050 ceph-mon[29546]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T21:04:25.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:24 smithi092 ceph-mon[34415]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T21:04:25.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:24 smithi142 ceph-mon[34846]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T21:04:26.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:25 smithi050 ceph-mon[29546]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T21:04:26.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:25 smithi050 ceph-mon[29546]: pgmap v324: 97 pgs: 49 active+clean, 12 creating+peering, 36 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:26.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:25 smithi092 ceph-mon[34415]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T21:04:26.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:25 smithi092 ceph-mon[34415]: pgmap v324: 97 pgs: 49 active+clean, 12 creating+peering, 36 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:26.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:25 smithi142 ceph-mon[34846]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T21:04:26.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:25 smithi142 ceph-mon[34846]: pgmap v324: 97 pgs: 49 active+clean, 12 creating+peering, 36 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:04:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.440+0000 7f6c85000700 1 -- 172.21.15.50:0/1003006626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80104400 msgr2=0x7f6c80104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.440+0000 7f6c85000700 1 --2- 172.21.15.50:0/1003006626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80104400 0x7f6c80104810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009990 tx=0x7f6c70009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 -- 172.21.15.50:0/1003006626 shutdown_connections 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 --2- 172.21.15.50:0/1003006626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80104400 0x7f6c80104810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 --2- 172.21.15.50:0/1003006626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80109680 0x7f6c80110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 --2- 172.21.15.50:0/1003006626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 0x7f6c80109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 -- 172.21.15.50:0/1003006626 >> 172.21.15.50:0/1003006626 conn(0x7f6c80079d10 msgr2=0x7f6c8007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 -- 172.21.15.50:0/1003006626 shutdown_connections 2024-05-10T21:04:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.441+0000 7f6c85000700 1 -- 172.21.15.50:0/1003006626 wait complete. 2024-05-10T21:04:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.442+0000 7f6c85000700 1 Processor -- start 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.442+0000 7f6c85000700 1 -- start start 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80104400 0x7f6c8019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 0x7f6c8019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80114320 con 0x7f6c80108d30 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801141a0 con 0x7f6c80109680 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c85000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801144a0 con 0x7f6c80104400 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c76a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 0x7f6c8019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c7f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80104400 0x7f6c8019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:45154/0 (socket says 172.21.15.50:45154) 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.443+0000 7f6c7fe00700 1 -- 172.21.15.50:0/100669470 learned_addr learned my addr 172.21.15.50:0/100669470 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 -- 172.21.15.50:0/100669470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80104400 msgr2=0x7f6c8019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80104400 0x7f6c8019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 -- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 msgr2=0x7f6c8019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 0x7f6c8019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 -- 172.21.15.50:0/100669470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c801a1ea0 con 0x7f6c80109680 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7fe00700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800ef30 tx=0x7f6c7800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780092e0 con 0x7f6c80109680 2024-05-10T21:04:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c78004bb0 con 0x7f6c80109680 2024-05-10T21:04:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c801a2130 con 0x7f6c80109680 2024-05-10T21:04:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.444+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78004020 con 0x7f6c80109680 2024-05-10T21:04:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.445+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c801a9ab0 con 0x7f6c80109680 2024-05-10T21:04:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.446+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c801058b0 con 0x7f6c80109680 2024-05-10T21:04:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.448+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6c7801b030 con 0x7f6c80109680 2024-05-10T21:04:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.450+0000 7f6c7cc00700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 0x7f6c640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.450+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f6c780e24a0 con 0x7f6c80109680 2024-05-10T21:04:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.450+0000 7f6c7f400700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 0x7f6c640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.451+0000 7f6c7f400700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 0x7f6c640b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b790 tx=0x7f6c7000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:26.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.452+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c78022020 con 0x7f6c80109680 2024-05-10T21:04:26.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.740+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6c800512c0 con 0x7f6c80109680 2024-05-10T21:04:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.945+0000 7f6c7cc00700 1 -- 172.21.15.50:0/100669470 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6c78010040 con 0x7f6c80109680 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.952+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 msgr2=0x7f6c640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.952+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 0x7f6c640b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b790 tx=0x7f6c7000b260 comp rx=0 tx=0).stop 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.952+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 msgr2=0x7f6c801a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.952+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800ef30 tx=0x7f6c7800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 shutdown_connections 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c640b2900 0x7f6c640b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c80109680 0x7f6c801a1800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80104400 0x7f6c8019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 --2- 172.21.15.50:0/100669470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c80108d30 0x7f6c8019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:26.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.954+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 >> 172.21.15.50:0/100669470 conn(0x7f6c80079d10 msgr2=0x7f6c80102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:26.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.955+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 shutdown_connections 2024-05-10T21:04:26.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:26.955+0000 7f6c85000700 1 -- 172.21.15.50:0/100669470 wait complete. 2024-05-10T21:04:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T21:04:26.995 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:26 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/100669470' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:26.995 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:26 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:26.995 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:26 smithi050 ceph-mon[29546]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T21:04:26.996 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:26 smithi050 ceph-mon[29546]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:27.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:26 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/100669470' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:27.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:26 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:27.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:26 smithi092 ceph-mon[34415]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T21:04:27.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:26 smithi092 ceph-mon[34415]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:27.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:26 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/100669470' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:27.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:26 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T21:04:27.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:26 smithi142 ceph-mon[34846]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T21:04:27.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:26 smithi142 ceph-mon[34846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T21:04:27.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:04:28.191 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:28.349 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:27 smithi050 ceph-mon[29546]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T21:04:28.349 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:27 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T21:04:28.349 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:27 smithi050 ceph-mon[29546]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T21:04:28.349 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:27 smithi050 ceph-mon[29546]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T21:04:28.349 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:27 smithi050 ceph-mon[29546]: pgmap v326: 97 pgs: 81 active+clean, 12 creating+peering, 4 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 657 B/s wr, 1 op/s 2024-05-10T21:04:28.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:27 smithi092 ceph-mon[34415]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T21:04:28.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:27 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T21:04:28.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:27 smithi092 ceph-mon[34415]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T21:04:28.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:27 smithi092 ceph-mon[34415]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T21:04:28.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:27 smithi092 ceph-mon[34415]: pgmap v326: 97 pgs: 81 active+clean, 12 creating+peering, 4 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 657 B/s wr, 1 op/s 2024-05-10T21:04:28.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:27 smithi142 ceph-mon[34846]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T21:04:28.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:27 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T21:04:28.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:27 smithi142 ceph-mon[34846]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T21:04:28.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:27 smithi142 ceph-mon[34846]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T21:04:28.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:27 smithi142 ceph-mon[34846]: pgmap v326: 97 pgs: 81 active+clean, 12 creating+peering, 4 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 657 B/s wr, 1 op/s 2024-05-10T21:04:29.368 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:28 smithi050 ceph-mon[29546]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T21:04:29.369 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:28 smithi050 ceph-mon[29546]: Cluster is now healthy 2024-05-10T21:04:29.369 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:28 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:active 2024-05-10T21:04:29.369 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:28 smithi050 ceph-mon[29546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.369 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:28 smithi050 ceph-mon[29546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:28 smithi092 ceph-mon[34415]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T21:04:29.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:28 smithi092 ceph-mon[34415]: Cluster is now healthy 2024-05-10T21:04:29.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:28 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:active 2024-05-10T21:04:29.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:28 smithi092 ceph-mon[34415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:28 smithi092 ceph-mon[34415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:28 smithi142 ceph-mon[34846]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T21:04:29.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:28 smithi142 ceph-mon[34846]: Cluster is now healthy 2024-05-10T21:04:29.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:28 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] up:active 2024-05-10T21:04:29.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:28 smithi142 ceph-mon[34846]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:28 smithi142 ceph-mon[34846]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:29.945 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: 2024-05-10T21:04:29.642+0000 7fd1deb7a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T21:04:30.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:29 smithi050 ceph-mon[29546]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 5 op/s 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.321+0000 7f0ebca00700 1 -- 172.21.15.50:0/417653145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb8110970 msgr2=0x7f0eb8110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.321+0000 7f0ebca00700 1 --2- 172.21.15.50:0/417653145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb8110970 0x7f0eb8110d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009990 tx=0x7f0ea8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 -- 172.21.15.50:0/417653145 shutdown_connections 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 --2- 172.21.15.50:0/417653145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb810b2e0 0x7f0eb8117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 --2- 172.21.15.50:0/417653145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb8110970 0x7f0eb8110d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 --2- 172.21.15.50:0/417653145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810a990 0x7f0eb810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 -- 172.21.15.50:0/417653145 >> 172.21.15.50:0/417653145 conn(0x7f0eb8077c80 msgr2=0x7f0eb80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:30.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 -- 172.21.15.50:0/417653145 shutdown_connections 2024-05-10T21:04:30.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.322+0000 7f0ebca00700 1 -- 172.21.15.50:0/417653145 wait complete. 2024-05-10T21:04:30.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.323+0000 7f0ebca00700 1 Processor -- start 2024-05-10T21:04:30.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 -- start start 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810b2e0 0x7f0eb81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 0x7f0eb81a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb811aac0 con 0x7f0eb810a990 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb811a940 con 0x7f0eb810b2e0 2024-05-10T21:04:30.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.324+0000 7f0ebca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb811ac40 con 0x7f0eb8110970 2024-05-10T21:04:30.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:30.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39268/0 (socket says 172.21.15.50:39268) 2024-05-10T21:04:30.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb5e00700 1 -- 172.21.15.50:0/1845577735 learned_addr learned my addr 172.21.15.50:0/1845577735 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:30.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 0x7f0eb81a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:30.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb5400700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810b2e0 0x7f0eb81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 -- 172.21.15.50:0/1845577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810b2e0 msgr2=0x7f0eb81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810b2e0 0x7f0eb81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 -- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 msgr2=0x7f0eb81a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb6800700 1 -- 172.21.15.50:0/1845577735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb81a6390 con 0x7f0eb8110970 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.325+0000 7f0eb5e00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0eb6800700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 0x7f0eb81a5cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000efc0 tx=0x7f0eb000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:30.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00092e0 con 0x7f0eb8110970 2024-05-10T21:04:30.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb0004bb0 con 0x7f0eb8110970 2024-05-10T21:04:30.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004020 con 0x7f0eb8110970 2024-05-10T21:04:30.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a6620 con 0x7f0eb8110970 2024-05-10T21:04:30.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.326+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a6b10 con 0x7f0eb8110970 2024-05-10T21:04:30.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.327+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb80512c0 con 0x7f0eb8110970 2024-05-10T21:04:30.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.330+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0eb001b030 con 0x7f0eb8110970 2024-05-10T21:04:30.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.331+0000 7f0eaea00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 0x7f0e9c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:30.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.332+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0eb00e2530 con 0x7f0eb8110970 2024-05-10T21:04:30.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.332+0000 7f0eb5e00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 0x7f0e9c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:30.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.332+0000 7f0eb5e00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 0x7f0e9c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8006010 tx=0x7f0ea800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:30.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.334+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0eb0022020 con 0x7f0eb8110970 2024-05-10T21:04:30.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:29 smithi092 ceph-mon[34415]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 5 op/s 2024-05-10T21:04:30.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:29 smithi142 ceph-mon[34846]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 5 op/s 2024-05-10T21:04:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.572+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eb802e5f0 con 0x7f0eb8110970 2024-05-10T21:04:30.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.574+0000 7f0eaea00700 1 -- 172.21.15.50:0/1845577735 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22170 (secure 0 0 0) 0x7f0eb0010040 con 0x7f0eb8110970 2024-05-10T21:04:30.579 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:04:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":80,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:26.742963+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:04:30.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 msgr2=0x7f0e9c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 0x7f0e9c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8006010 tx=0x7f0ea800b280 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 msgr2=0x7f0eb81a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 0x7f0eb81a5cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000efc0 tx=0x7f0eb000c3a0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 shutdown_connections 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0e9c0b2900 0x7f0e9c0b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb8110970 0x7f0eb81a5cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0eb810a990 0x7f0eb81a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 --2- 172.21.15.50:0/1845577735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0eb810b2e0 0x7f0eb81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.578+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 >> 172.21.15.50:0/1845577735 conn(0x7f0eb8077c80 msgr2=0x7f0eb807bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.579+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 shutdown_connections 2024-05-10T21:04:30.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:30.579+0000 7f0ebca00700 1 -- 172.21.15.50:0/1845577735 wait complete. 2024-05-10T21:04:31.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:30 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1845577735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:31.358 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:04:31.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:30 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1845577735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:31.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:30 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1845577735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:31.820 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:32.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:31 smithi050 ceph-mon[29546]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T21:04:32.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:31 smithi092 ceph-mon[34415]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T21:04:32.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:31 smithi142 ceph-mon[34846]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T21:04:33.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 -- 172.21.15.50:0/4291112810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb40810b8a0 msgr2=0x7fb408113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 --2- 172.21.15.50:0/4291112810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb40810b8a0 0x7fb408113200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb400009f90 tx=0x7fb40000b330 comp rx=0 tx=0).stop 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 -- 172.21.15.50:0/4291112810 shutdown_connections 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 --2- 172.21.15.50:0/4291112810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb40810b8a0 0x7fb408113200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 --2- 172.21.15.50:0/4291112810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb408106620 0x7fb408106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 --2- 172.21.15.50:0/4291112810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 0x7fb40810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 -- 172.21.15.50:0/4291112810 >> 172.21.15.50:0/4291112810 conn(0x7fb408102030 msgr2=0x7fb408104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 -- 172.21.15.50:0/4291112810 shutdown_connections 2024-05-10T21:04:33.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.990+0000 7fb40e200700 1 -- 172.21.15.50:0/4291112810 wait complete. 2024-05-10T21:04:33.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.991+0000 7fb40e200700 1 Processor -- start 2024-05-10T21:04:33.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.991+0000 7fb40e200700 1 -- start start 2024-05-10T21:04:33.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:33.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 0x7fb40819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40810b8a0 0x7fb4081a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081164d0 con 0x7fb40810b8a0 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb408116350 con 0x7fb408106620 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb408116650 con 0x7fb40810af50 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40d800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40810b8a0 0x7fb4081a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb407e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 0x7fb40819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36610/0 (socket says 172.21.15.50:36610) 2024-05-10T21:04:33.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.992+0000 7fb40ce00700 1 -- 172.21.15.50:0/2284119134 learned_addr learned my addr 172.21.15.50:0/2284119134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:33.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 -- 172.21.15.50:0/2284119134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 msgr2=0x7fb40819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:33.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 0x7fb40819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:33.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 -- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40810b8a0 msgr2=0x7fb4081a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:33.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40810b8a0 0x7fb4081a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 -- 172.21.15.50:0/2284119134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4081a1cb0 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.993+0000 7fb40ce00700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e9c0 tx=0x7fb3fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.994+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc004a30 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.994+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc00cc70 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.994+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4081a1fa0 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.994+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc005490 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.994+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a98c0 con 0x7fb408106620 2024-05-10T21:04:33.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.995+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080512c0 con 0x7fb408106620 2024-05-10T21:04:34.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.997+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb3fc004bd0 con 0x7fb408106620 2024-05-10T21:04:34.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.999+0000 7fb405600700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:34.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.999+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb3fc0e2230 con 0x7fb408106620 2024-05-10T21:04:34.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:33.999+0000 7fb407e00700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:34.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.000+0000 7fb407e00700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 0x7fb3f00b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb408105a60 tx=0x7fb3f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:34.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.003+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3fc0aa000 con 0x7fb408106620 2024-05-10T21:04:34.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.239+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb40802e5f0 con 0x7fb408106620 2024-05-10T21:04:34.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.241+0000 7fb405600700 1 -- 172.21.15.50:0/2284119134 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22170 (secure 0 0 0) 0x7fb3fc0a9900 con 0x7fb408106620 2024-05-10T21:04:34.245 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:04:34.246 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":80,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:26.742963+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.244+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 msgr2=0x7fb3f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 0x7fb3f00b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb408105a60 tx=0x7fb3f4009260 comp rx=0 tx=0).stop 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 msgr2=0x7fb40819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e9c0 tx=0x7fb3fc00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 shutdown_connections 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb408106620 0x7fb40819cde0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:34.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40810b8a0 0x7fb4081a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:34.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 --2- 172.21.15.50:0/2284119134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40810af50 0x7fb40819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:34.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 >> 172.21.15.50:0/2284119134 conn(0x7fb408102030 msgr2=0x7fb40810a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:34.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 shutdown_connections 2024-05-10T21:04:34.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:34.245+0000 7fb40e200700 1 -- 172.21.15.50:0/2284119134 wait complete. 2024-05-10T21:04:34.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:34 smithi050 ceph-mon[29546]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-10T21:04:34.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:34 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2284119134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:35.016 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-10T21:04:35.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:34 smithi092 ceph-mon[34415]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-10T21:04:35.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:34 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2284119134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:35.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:34 smithi142 ceph-mon[34846]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-10T21:04:35.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:34 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2284119134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:04:35.494 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:36.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:36 smithi050 ceph-mon[29546]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-10T21:04:37.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:36 smithi092 ceph-mon[34415]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-10T21:04:37.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:36 smithi142 ceph-mon[34846]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-10T21:04:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 -- 172.21.15.50:0/248674551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c108f00 msgr2=0x7fc78c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 --2- 172.21.15.50:0/248674551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c108f00 0x7fc78c109310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7780099c0 tx=0x7fc778009cd0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 -- 172.21.15.50:0/248674551 shutdown_connections 2024-05-10T21:04:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 --2- 172.21.15.50:0/248674551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c109850 0x7fc78c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 --2- 172.21.15.50:0/248674551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 --2- 172.21.15.50:0/248674551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c108f00 0x7fc78c109310 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 -- 172.21.15.50:0/248674551 >> 172.21.15.50:0/248674551 conn(0x7fc78c0ffea0 msgr2=0x7fc78c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 -- 172.21.15.50:0/248674551 shutdown_connections 2024-05-10T21:04:37.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.695+0000 7fc793a00700 1 -- 172.21.15.50:0/248674551 wait complete. 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.696+0000 7fc793a00700 1 Processor -- start 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 -- start start 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c109850 0x7fc78c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1144b0 con 0x7fc78c109850 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c114330 con 0x7fc78c108f00 2024-05-10T21:04:37.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc793a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c114630 con 0x7fc78c1045d0 2024-05-10T21:04:37.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc78be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc78be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36626/0 (socket says 172.21.15.50:36626) 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc78be00700 1 -- 172.21.15.50:0/1399269663 learned_addr learned my addr 172.21.15.50:0/1399269663 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc791a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c109850 0x7fc78c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.697+0000 7fc791000700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 -- 172.21.15.50:0/1399269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 msgr2=0x7fc78c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 -- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c109850 msgr2=0x7fc78c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c109850 0x7fc78c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 -- 172.21.15.50:0/1399269663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1a91e0 con 0x7fc78c108f00 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc791000700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.698+0000 7fc78be00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b580 tx=0x7fc778004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.699+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778009e60 con 0x7fc78c108f00 2024-05-10T21:04:37.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.699+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc778003c70 con 0x7fc78c108f00 2024-05-10T21:04:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.699+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1a9470 con 0x7fc78c108f00 2024-05-10T21:04:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.699+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004910 con 0x7fc78c108f00 2024-05-10T21:04:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.699+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a9860 con 0x7fc78c108f00 2024-05-10T21:04:37.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.700+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c0512c0 con 0x7fc78c108f00 2024-05-10T21:04:37.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.702+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc7780037c0 con 0x7fc78c108f00 2024-05-10T21:04:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.704+0000 7fc789600700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 0x7fc7740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.704+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc7780e2a50 con 0x7fc78c108f00 2024-05-10T21:04:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.704+0000 7fc791000700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 0x7fc7740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:37.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.705+0000 7fc791000700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 0x7fc7740b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc780011490 tx=0x7fc780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:37.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.708+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc778013070 con 0x7fc78c108f00 2024-05-10T21:04:38.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:37.997+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc78c02e7b0 con 0x7fc78c108f00 2024-05-10T21:04:38.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.651+0000 7fc789600700 1 -- 172.21.15.50:0/1399269663 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc7780aa8b0 con 0x7fc78c108f00 2024-05-10T21:04:38.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 msgr2=0x7fc7740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:38.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 0x7fc7740b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc780011490 tx=0x7fc780009260 comp rx=0 tx=0).stop 2024-05-10T21:04:38.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 msgr2=0x7fc78c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:38.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b580 tx=0x7fc778004270 comp rx=0 tx=0).stop 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 shutdown_connections 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7740b2880 0x7fc7740b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc78c109850 0x7fc78c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c1045d0 0x7fc78c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.656+0000 7fc793a00700 1 --2- 172.21.15.50:0/1399269663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c108f00 0x7fc78c077110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.657+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 >> 172.21.15.50:0/1399269663 conn(0x7fc78c0ffea0 msgr2=0x7fc78c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.657+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 shutdown_connections 2024-05-10T21:04:38.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:38.657+0000 7fc793a00700 1 -- 172.21.15.50:0/1399269663 wait complete. 2024-05-10T21:04:38.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:38 smithi050 ceph-mon[29546]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-10T21:04:38.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:38 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1399269663' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:38.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:38 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:38.935 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:38 smithi142 ceph-mon[34846]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-10T21:04:38.935 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:38 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1399269663' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:38.935 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:38 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:39.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:38 smithi092 ceph-mon[34415]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-10T21:04:39.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:38 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1399269663' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:39.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:38 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T21:04:39.468 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-10T21:04:39.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-10T21:04:39.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:39 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T21:04:39.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:39 smithi050 ceph-mon[29546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:39.948 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:40.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:39 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T21:04:40.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:39 smithi092 ceph-mon[34415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:40.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:39 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T21:04:40.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:39 smithi142 ceph-mon[34846]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:40.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:40 smithi050 ceph-mon[29546]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-10T21:04:41.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:40 smithi092 ceph-mon[34415]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-10T21:04:41.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:40 smithi142 ceph-mon[34846]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-05-10T21:04:42.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.120+0000 7f4975600700 1 -- 172.21.15.50:0/3875154035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 msgr2=0x7f4970077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:42.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.120+0000 7f4975600700 1 --2- 172.21.15.50:0/3875154035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f4970077250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4960009990 tx=0x7f4960009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 -- 172.21.15.50:0/3875154035 shutdown_connections 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 --2- 172.21.15.50:0/3875154035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 0x7f4970112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 --2- 172.21.15.50:0/3875154035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f4970077250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 --2- 172.21.15.50:0/3875154035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 0x7f4970110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 -- 172.21.15.50:0/3875154035 >> 172.21.15.50:0/3875154035 conn(0x7f49700ffea0 msgr2=0x7f49701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 -- 172.21.15.50:0/3875154035 shutdown_connections 2024-05-10T21:04:42.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.121+0000 7f4975600700 1 -- 172.21.15.50:0/3875154035 wait complete. 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.122+0000 7f4975600700 1 Processor -- start 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.122+0000 7f4975600700 1 -- start start 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 0x7f497019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 0x7f49701a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49701166c0 con 0x7f4970110b80 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4970116540 con 0x7f4970077790 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f4975600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4970116840 con 0x7f4970076e40 2024-05-10T21:04:42.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 0x7f497019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50902/0 (socket says 172.21.15.50:50902) 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 -- 172.21.15.50:0/3994103818 learned_addr learned my addr 172.21.15.50:0/3994103818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496f400700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 0x7f49701a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 -- 172.21.15.50:0/3994103818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 msgr2=0x7f497019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 0x7f497019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 -- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 msgr2=0x7f49701a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 0x7f49701a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.123+0000 7f496ea00700 1 -- 172.21.15.50:0/3994103818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49701a1ee0 con 0x7f4970076e40 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f496ea00700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4960005910 tx=0x7f496000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960027070 con 0x7f4970076e40 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49701a2170 con 0x7f4970076e40 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960022470 con 0x7f4970076e40 2024-05-10T21:04:42.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960017650 con 0x7f4970076e40 2024-05-10T21:04:42.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.124+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701a2650 con 0x7f4970076e40 2024-05-10T21:04:42.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.125+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49700512c0 con 0x7f4970076e40 2024-05-10T21:04:42.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.128+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f496002a070 con 0x7f4970076e40 2024-05-10T21:04:42.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.130+0000 7f4967400700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 0x7f49540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:42.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.130+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f49600e30b0 con 0x7f4970076e40 2024-05-10T21:04:42.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.130+0000 7f496e000700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 0x7f49540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:42.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.131+0000 7f496e000700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 0x7f49540b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4970103340 tx=0x7f495c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:42.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.132+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4960013070 con 0x7f4970076e40 2024-05-10T21:04:42.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.420+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f497019e3a0 con 0x7f4970076e40 2024-05-10T21:04:42.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.781+0000 7f4967400700 1 -- 172.21.15.50:0/3994103818 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f497019e3a0 con 0x7f4970076e40 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 msgr2=0x7f49540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 0x7f49540b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4970103340 tx=0x7f495c009260 comp rx=0 tx=0).stop 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 msgr2=0x7f497019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4960005910 tx=0x7f496000b670 comp rx=0 tx=0).stop 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 shutdown_connections 2024-05-10T21:04:42.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2880 0x7f49540b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970110b80 0x7f49701a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076e40 0x7f497019d010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 --2- 172.21.15.50:0/3994103818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077790 0x7f497019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.785+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 >> 172.21.15.50:0/3994103818 conn(0x7f49700ffea0 msgr2=0x7f49701004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.786+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 shutdown_connections 2024-05-10T21:04:42.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:42.786+0000 7f4975600700 1 -- 172.21.15.50:0/3994103818 wait complete. 2024-05-10T21:04:43.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:42 smithi092 ceph-mon[34415]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-10T21:04:43.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:42 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3994103818' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:42 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:42 smithi142 ceph-mon[34846]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-10T21:04:43.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:42 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3994103818' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:42 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:42 smithi050 ceph-mon[29546]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 0 op/s 2024-05-10T21:04:43.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:42 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3994103818' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:42 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T21:04:43.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-10T21:04:43.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T21:04:43.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T21:04:43.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T21:04:43.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:43.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-10T21:04:43.922 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:43 smithi050 ceph-mon[29546]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-10T21:04:44.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:43 smithi092 ceph-mon[34415]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-10T21:04:44.126 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-10T21:04:44.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:43 smithi142 ceph-mon[34846]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: Cluster is now healthy 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-10T21:04:45.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-10T21:04:45.096 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:44 smithi050 ceph-mon[29546]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-10T21:04:45.122 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: 2024-05-10T21:04:44.643+0000 7fd1deb7a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-10T21:04:45.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T21:04:45.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:04:45.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T21:04:45.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T21:04:45.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: Cluster is now healthy 2024-05-10T21:04:45.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-10T21:04:45.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-10T21:04:45.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:44 smithi092 ceph-mon[34415]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: Cluster is now healthy 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-10T21:04:45.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:44 smithi142 ceph-mon[34846]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-10T21:04:46.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:45 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:active 2024-05-10T21:04:46.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:45 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-10T21:04:46.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:45 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:active 2024-05-10T21:04:46.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:45 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-10T21:04:46.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:45 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] up:active 2024-05-10T21:04:46.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:45 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-10T21:04:46.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 -- 172.21.15.50:0/795987816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d90110970 msgr2=0x7f3d90110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:46.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 --2- 172.21.15.50:0/795987816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d90110970 0x7f3d90110d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009990 tx=0x7f3d80009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 -- 172.21.15.50:0/795987816 shutdown_connections 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 --2- 172.21.15.50:0/795987816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d90110970 0x7f3d90110d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 --2- 172.21.15.50:0/795987816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010b2e0 0x7f3d90117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 --2- 172.21.15.50:0/795987816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d9010a990 0x7f3d9010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 -- 172.21.15.50:0/795987816 >> 172.21.15.50:0/795987816 conn(0x7f3d90077c80 msgr2=0x7f3d900780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 -- 172.21.15.50:0/795987816 shutdown_connections 2024-05-10T21:04:46.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.390+0000 7f3d96200700 1 -- 172.21.15.50:0/795987816 wait complete. 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.391+0000 7f3d96200700 1 Processor -- start 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 -- start start 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 0x7f3d901a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 0x7f3d901a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011aac0 con 0x7f3d90110970 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011a940 con 0x7f3d9010b2e0 2024-05-10T21:04:46.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d96200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011ac40 con 0x7f3d9010a990 2024-05-10T21:04:46.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d8f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:46.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.392+0000 7f3d8fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 0x7f3d901a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:46.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d8f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59042/0 (socket says 172.21.15.50:59042) 2024-05-10T21:04:46.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d8f400700 1 -- 172.21.15.50:0/1274588054 learned_addr learned my addr 172.21.15.50:0/1274588054 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:46.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 0x7f3d901a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:46.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 -- 172.21.15.50:0/1274588054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 msgr2=0x7f3d901a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:46.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 -- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 msgr2=0x7f3d901a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:46.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 0x7f3d901a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 -- 172.21.15.50:0/1274588054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901a6320 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d8fe00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 0x7f3d901a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d8f400700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.393+0000 7f3d86a00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 0x7f3d901a19f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800e9c0 tx=0x7f3d7800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.394+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800c9c0 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.394+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901a6610 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.394+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7800cb60 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.394+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78018680 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.394+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901adf30 con 0x7f3d9010b2e0 2024-05-10T21:04:46.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.395+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9010c080 con 0x7f3d9010b2e0 2024-05-10T21:04:46.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.398+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3d78010430 con 0x7f3d9010b2e0 2024-05-10T21:04:46.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.399+0000 7f3d8cc00700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 0x7f3d740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:46.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.400+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3d780e22a0 con 0x7f3d9010b2e0 2024-05-10T21:04:46.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.400+0000 7f3d8f400700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 0x7f3d740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:46.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.400+0000 7f3d8f400700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 0x7f3d740b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000b2e0 tx=0x7f3d80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:46.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.403+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d780aa070 con 0x7f3d9010b2e0 2024-05-10T21:04:46.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.691+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3d901a27b0 con 0x7f3d9010b2e0 2024-05-10T21:04:46.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.770+0000 7f3d8cc00700 1 -- 172.21.15.50:0/1274588054 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f3d780b20f0 con 0x7f3d9010b2e0 2024-05-10T21:04:46.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 msgr2=0x7f3d740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:46.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 0x7f3d740b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000b2e0 tx=0x7f3d80005fd0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 msgr2=0x7f3d901a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:46.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 0x7f3d901a19f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800e9c0 tx=0x7f3d7800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 shutdown_connections 2024-05-10T21:04:46.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3d740b2880 0x7f3d740b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d90110970 0x7f3d901a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9010a990 0x7f3d901a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 --2- 172.21.15.50:0/1274588054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d9010b2e0 0x7f3d901a19f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:46.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.774+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 >> 172.21.15.50:0/1274588054 conn(0x7f3d90077c80 msgr2=0x7f3d9010e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:46.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.775+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 shutdown_connections 2024-05-10T21:04:46.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:46.775+0000 7f3d96200700 1 -- 172.21.15.50:0/1274588054 wait complete. 2024-05-10T21:04:47.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:46 smithi092 ceph-mon[34415]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-10T21:04:47.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:46 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1274588054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:46 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:46 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T21:04:47.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:46 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:47.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:46 smithi142 ceph-mon[34846]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-10T21:04:47.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:46 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1274588054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:46 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:46 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T21:04:47.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:46 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:47.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:46 smithi050 ceph-mon[29546]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-10T21:04:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:46 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1274588054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:46 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T21:04:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:46 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T21:04:47.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:46 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:47.574 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-10T21:04:48.031 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:49.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:48 smithi092 ceph-mon[34415]: pgmap v336: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:04:49.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:48 smithi142 ceph-mon[34846]: pgmap v336: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:04:49.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:48 smithi050 ceph-mon[29546]: pgmap v336: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.307+0000 7fa4afe00700 1 -- 172.21.15.50:0/3076282744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b0075d30 msgr2=0x7fa4b0076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.307+0000 7fa4afe00700 1 --2- 172.21.15.50:0/3076282744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b0075d30 0x7fa4b0076140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b0a0 tx=0x7fa4a000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 -- 172.21.15.50:0/3076282744 shutdown_connections 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 --2- 172.21.15.50:0/3076282744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b01109b0 0x7fa4b0112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 --2- 172.21.15.50:0/3076282744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b0075d30 0x7fa4b0076140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 --2- 172.21.15.50:0/3076282744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 0x7fa4b0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 -- 172.21.15.50:0/3076282744 >> 172.21.15.50:0/3076282744 conn(0x7fa4b0079d10 msgr2=0x7fa4b007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:50.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 -- 172.21.15.50:0/3076282744 shutdown_connections 2024-05-10T21:04:50.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.308+0000 7fa4afe00700 1 -- 172.21.15.50:0/3076282744 wait complete. 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.309+0000 7fa4afe00700 1 Processor -- start 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.309+0000 7fa4afe00700 1 -- start start 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 0x7fa4b019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b01109b0 0x7fa4b01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01164c0 con 0x7fa4b01109b0 2024-05-10T21:04:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0116340 con 0x7fa4b0075d30 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4afe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0116640 con 0x7fa4b0076680 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4aea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4aea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43890/0 (socket says 172.21.15.50:43890) 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4aea00700 1 -- 172.21.15.50:0/4028415303 learned_addr learned my addr 172.21.15.50:0/4028415303 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4af400700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b01109b0 0x7fa4b01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.310+0000 7fa4ae000700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 0x7fa4b019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 -- 172.21.15.50:0/4028415303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 msgr2=0x7fa4b019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 0x7fa4b019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 -- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b01109b0 msgr2=0x7fa4b01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b01109b0 0x7fa4b01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 -- 172.21.15.50:0/4028415303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b01a1d20 con 0x7fa4b0075d30 2024-05-10T21:04:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4aea00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b070 tx=0x7fa4a0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0021070 con 0x7fa4b0075d30 2024-05-10T21:04:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a0007e30 con 0x7fa4b0075d30 2024-05-10T21:04:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00046c0 con 0x7fa4b0075d30 2024-05-10T21:04:50.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.311+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b01a1fb0 con 0x7fa4b0075d30 2024-05-10T21:04:50.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.312+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01a24a0 con 0x7fa4b0075d30 2024-05-10T21:04:50.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.313+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00772a0 con 0x7fa4b0075d30 2024-05-10T21:04:50.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.315+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa4a0007980 con 0x7fa4b0075d30 2024-05-10T21:04:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.317+0000 7fa4a7400700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 0x7fa4940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.317+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa4a000f020 con 0x7fa4b0075d30 2024-05-10T21:04:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.317+0000 7fa4ae000700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 0x7fa4940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.318+0000 7fa4ae000700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 0x7fa4940b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4b007a790 tx=0x7fa49c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.319+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4a002a020 con 0x7fa4b0075d30 2024-05-10T21:04:50.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.605+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa4b00512c0 con 0x7fa4b0075d30 2024-05-10T21:04:50.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.837+0000 7fa4a7400700 1 -- 172.21.15.50:0/4028415303 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fa4a00ab7f0 con 0x7fa4b0075d30 2024-05-10T21:04:50.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 msgr2=0x7fa4940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:50.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 0x7fa4940b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4b007a790 tx=0x7fa49c009260 comp rx=0 tx=0).stop 2024-05-10T21:04:50.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 msgr2=0x7fa4b019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:50.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b070 tx=0x7fa4a0004d00 comp rx=0 tx=0).stop 2024-05-10T21:04:50.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 shutdown_connections 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa4940b2880 0x7fa4940b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4b0075d30 0x7fa4b019ce50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa4b01109b0 0x7fa4b01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 --2- 172.21.15.50:0/4028415303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4b0076680 0x7fa4b019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 >> 172.21.15.50:0/4028415303 conn(0x7fa4b0079d10 msgr2=0x7fa4b0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.842+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 shutdown_connections 2024-05-10T21:04:50.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:50.843+0000 7fa4afe00700 1 -- 172.21.15.50:0/4028415303 wait complete. 2024-05-10T21:04:50.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:50 smithi050 ceph-mon[29546]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T21:04:50.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:50 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4028415303' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:50.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:50 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:51.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:50 smithi092 ceph-mon[34415]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T21:04:51.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:50 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4028415303' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:51.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:50 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:51.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:50 smithi142 ceph-mon[34846]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T21:04:51.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:50 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4028415303' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:51.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:50 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T21:04:51.648 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-10T21:04:51.649 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-10T21:04:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:51 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:51 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:52.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:51 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T21:04:52.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:51 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:52.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:51 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T21:04:52.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:51 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:04:53.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:52 smithi092 ceph-mon[34415]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-10T21:04:53.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:52 smithi142 ceph-mon[34846]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-10T21:04:53.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:52 smithi050 ceph-mon[29546]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 -- 172.21.15.50:0/3209046737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc077790 msgr2=0x7f09bc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 --2- 172.21.15.50:0/3209046737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc077790 0x7f09bc110640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009990 tx=0x7f09a8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 -- 172.21.15.50:0/3209046737 shutdown_connections 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 --2- 172.21.15.50:0/3209046737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc110b80 0x7f09bc112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 --2- 172.21.15.50:0/3209046737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc076e40 0x7f09bc077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 --2- 172.21.15.50:0/3209046737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc077790 0x7f09bc110640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 -- 172.21.15.50:0/3209046737 >> 172.21.15.50:0/3209046737 conn(0x7f09bc0ffea0 msgr2=0x7f09bc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:54.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 -- 172.21.15.50:0/3209046737 shutdown_connections 2024-05-10T21:04:54.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.431+0000 7f09c2c00700 1 -- 172.21.15.50:0/3209046737 wait complete. 2024-05-10T21:04:54.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 Processor -- start 2024-05-10T21:04:54.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 -- start start 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc077790 0x7f09bc19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc110b80 0x7f09bc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc1166d0 con 0x7f09bc076e40 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc116550 con 0x7f09bc077790 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.433+0000 7f09c2c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc116850 con 0x7f09bc110b80 2024-05-10T21:04:54.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:54.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54306/0 (socket says 172.21.15.50:54306) 2024-05-10T21:04:54.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 -- 172.21.15.50:0/2300664835 learned_addr learned my addr 172.21.15.50:0/2300664835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:54.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09c0c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc110b80 0x7f09bc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:54.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bb400700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc077790 0x7f09bc19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 -- 172.21.15.50:0/2300664835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc110b80 msgr2=0x7f09bc1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc110b80 0x7f09bc1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 -- 172.21.15.50:0/2300664835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc077790 msgr2=0x7f09bc19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc077790 0x7f09bc19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.434+0000 7f09bbe00700 1 -- 172.21.15.50:0/2300664835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc1a1f50 con 0x7f09bc076e40 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09bbe00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00eb30 tx=0x7f09ac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00cbd0 con 0x7f09bc076e40 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ac0044d0 con 0x7f09bc076e40 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc1a2240 con 0x7f09bc076e40 2024-05-10T21:04:54.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac003e30 con 0x7f09bc076e40 2024-05-10T21:04:54.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.435+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc1a9b60 con 0x7f09bc076e40 2024-05-10T21:04:54.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.436+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09bc0512c0 con 0x7f09bc076e40 2024-05-10T21:04:54.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.439+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09ac004000 con 0x7f09bc076e40 2024-05-10T21:04:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.441+0000 7f09b8c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 0x7f09a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.441+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f09ac0e2680 con 0x7f09bc076e40 2024-05-10T21:04:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.441+0000 7f09bb400700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 0x7f09a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:54.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.442+0000 7f09bb400700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 0x7f09a00b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09bc103400 tx=0x7f09a800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:54.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.444+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09ac054df0 con 0x7f09bc076e40 2024-05-10T21:04:54.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.717+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f09bc063c00 con 0x7f09a00b2900 2024-05-10T21:04:54.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.754+0000 7f09b8c00700 1 -- 172.21.15.50:0/2300664835 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09bc063c00 con 0x7f09a00b2900 2024-05-10T21:04:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 msgr2=0x7f09a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 0x7f09a00b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09bc103400 tx=0x7f09a800b260 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 msgr2=0x7f09bc19d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00eb30 tx=0x7f09ac00ee40 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 shutdown_connections 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09a00b2900 0x7f09a00b4e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09bc110b80 0x7f09bc1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09bc076e40 0x7f09bc19d080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 --2- 172.21.15.50:0/2300664835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09bc077790 0x7f09bc19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 >> 172.21.15.50:0/2300664835 conn(0x7f09bc0ffea0 msgr2=0x7f09bc101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 shutdown_connections 2024-05-10T21:04:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:54.757+0000 7f09c2c00700 1 -- 172.21.15.50:0/2300664835 wait complete. 2024-05-10T21:04:55.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:54 smithi092 ceph-mon[34415]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T21:04:55.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:54 smithi142 ceph-mon[34846]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T21:04:55.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:54 smithi050 ceph-mon[29546]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T21:04:55.171 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:04:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:04:54.726+0000 7fc323800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T21:04:55.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:04:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:04:54.726+0000 7fc323800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T21:04:55.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:04:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:04:54.726+0000 7fc323800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T21:04:55.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:04:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:04:54.726+0000 7fc323800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T21:04:55.172 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:04:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:04:54.726+0000 7fc323800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T21:04:55.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T21:04:56.166 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:04:56.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:56.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-05-10T21:04:56.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T21:04:56.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-05-10T21:04:56.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-10T21:04:56.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-10T21:04:56.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-10T21:04:56.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:56.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-05-10T21:04:56.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T21:04:56.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-05-10T21:04:56.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-10T21:04:56.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-10T21:04:56.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-10T21:04:56.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-10T21:04:57.154 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-10T21:04:57.154 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-05-10T21:04:57.154 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T21:04:57.154 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-05-10T21:04:57.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-10T21:04:57.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-10T21:04:57.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-10T21:04:57.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T21:04:57.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:56 smithi050 ceph-mon[29546]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-05-10T21:04:57.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-10T21:04:57.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-05-10T21:04:57.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T21:04:57.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-05-10T21:04:57.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-10T21:04:57.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-10T21:04:57.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-10T21:04:57.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T21:04:57.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:56 smithi092 ceph-mon[34415]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-10T21:04:57.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-10T21:04:57.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T21:04:57.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:56 smithi142 ceph-mon[34846]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-05-10T21:04:58.104 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:57 smithi050 ceph-mon[29546]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T21:04:58.104 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:57 smithi050 ceph-mon[29546]: pgmap v343: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T21:04:58.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.357+0000 7f823ca00700 1 -- 172.21.15.50:0/2978881445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238110970 msgr2=0x7f8238110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.357+0000 7f823ca00700 1 --2- 172.21.15.50:0/2978881445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238110970 0x7f8238110d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8228009990 tx=0x7f8228009ca0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 -- 172.21.15.50:0/2978881445 shutdown_connections 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 --2- 172.21.15.50:0/2978881445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f823810b2e0 0x7f8238117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 --2- 172.21.15.50:0/2978881445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238110970 0x7f8238110d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 --2- 172.21.15.50:0/2978881445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810a990 0x7f823810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 -- 172.21.15.50:0/2978881445 >> 172.21.15.50:0/2978881445 conn(0x7f8238077c80 msgr2=0x7f82380780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 -- 172.21.15.50:0/2978881445 shutdown_connections 2024-05-10T21:04:58.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.358+0000 7f823ca00700 1 -- 172.21.15.50:0/2978881445 wait complete. 2024-05-10T21:04:58.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.359+0000 7f823ca00700 1 Processor -- start 2024-05-10T21:04:58.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.360+0000 7f823ca00700 1 -- start start 2024-05-10T21:04:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.360+0000 7f823ca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.360+0000 7f823ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b2e0 0x7f82381a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.360+0000 7f823ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238110970 0x7f82381a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f823ca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f823811aa70 con 0x7f823810a990 2024-05-10T21:04:58.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f823ca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f823811a8f0 con 0x7f8238110970 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f823ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f823811abf0 con 0x7f823810b2e0 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54346/0 (socket says 172.21.15.50:54346) 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 -- 172.21.15.50:0/1590842784 learned_addr learned my addr 172.21.15.50:0/1590842784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235400700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b2e0 0x7f82381a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:58.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8236800700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238110970 0x7f82381a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 -- 172.21.15.50:0/1590842784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b2e0 msgr2=0x7f82381a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b2e0 0x7f82381a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 -- 172.21.15.50:0/1590842784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238110970 msgr2=0x7f82381a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238110970 0x7f82381a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.361+0000 7f8235e00700 1 -- 172.21.15.50:0/1590842784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82381a6370 con 0x7f823810a990 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f8235e00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f8228009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228026070 con 0x7f823810a990 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8228021410 con 0x7f823810a990 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381a6600 con 0x7f823810a990 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228017650 con 0x7f823810a990 2024-05-10T21:04:58.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.362+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a6ae0 con 0x7f823810a990 2024-05-10T21:04:58.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.363+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380512c0 con 0x7f823810a990 2024-05-10T21:04:58.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.367+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8228029070 con 0x7f823810a990 2024-05-10T21:04:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.369+0000 7f822ea00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 0x7f821c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:04:58.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.369+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f82280e3390 con 0x7f823810a990 2024-05-10T21:04:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.369+0000 7f8235400700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 0x7f821c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:04:58.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:57 smithi092 ceph-mon[34415]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T21:04:58.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:57 smithi092 ceph-mon[34415]: pgmap v343: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T21:04:58.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.370+0000 7f8235400700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 0x7f821c0b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f823807b140 tx=0x7f8224009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:04:58.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.371+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8228013070 con 0x7f823810a990 2024-05-10T21:04:58.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:57 smithi142 ceph-mon[34846]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T21:04:58.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:57 smithi142 ceph-mon[34846]: pgmap v343: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T21:04:58.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.644+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f82381a2800 con 0x7f821c0b2910 2024-05-10T21:04:58.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.687+0000 7f822ea00700 1 -- 172.21.15.50:0/1590842784 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82381a2800 con 0x7f821c0b2910 2024-05-10T21:04:58.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.690+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 msgr2=0x7f821c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:58.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.690+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 0x7f821c0b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f823807b140 tx=0x7f8224009260 comp rx=0 tx=0).stop 2024-05-10T21:04:58.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.690+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 msgr2=0x7f82381a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:04:58.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.690+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f8228009510 comp rx=0 tx=0).stop 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 shutdown_connections 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f821c0b2910 0x7f821c0b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238110970 0x7f82381a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f823810a990 0x7f82381a14a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 --2- 172.21.15.50:0/1590842784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b2e0 0x7f82381a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 >> 172.21.15.50:0/1590842784 conn(0x7f8238077c80 msgr2=0x7f823807bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 shutdown_connections 2024-05-10T21:04:58.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:04:58.691+0000 7f823ca00700 1 -- 172.21.15.50:0/1590842784 wait complete. 2024-05-10T21:04:59.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:58 smithi050 ceph-mon[29546]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:59.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:58 smithi092 ceph-mon[34415]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:59.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:58 smithi142 ceph-mon[34846]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:04:59.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T21:04:59.946 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:00.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:04:59 smithi050 ceph-mon[29546]: pgmap v344: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:00.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:04:59 smithi092 ceph-mon[34415]: pgmap v344: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:00.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:04:59 smithi142 ceph-mon[34846]: pgmap v344: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 -- 172.21.15.50:0/2107842611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 msgr2=0x7fb6c8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 --2- 172.21.15.50:0/2107842611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c8076140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009990 tx=0x7fb6bc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 -- 172.21.15.50:0/2107842611 shutdown_connections 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 --2- 172.21.15.50:0/2107842611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 0x7fb6c8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 --2- 172.21.15.50:0/2107842611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c8076140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 --2- 172.21.15.50:0/2107842611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 0x7fb6c8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.199+0000 7fb6d0000700 1 -- 172.21.15.50:0/2107842611 >> 172.21.15.50:0/2107842611 conn(0x7fb6c8100040 msgr2=0x7fb6c8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:02.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.200+0000 7fb6d0000700 1 -- 172.21.15.50:0/2107842611 shutdown_connections 2024-05-10T21:05:02.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.200+0000 7fb6d0000700 1 -- 172.21.15.50:0/2107842611 wait complete. 2024-05-10T21:05:02.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.201+0000 7fb6d0000700 1 Processor -- start 2024-05-10T21:05:02.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 -- start start 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 0x7fb6c819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 0x7fb6c81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81166e0 con 0x7fb6c8076680 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8116560 con 0x7fb6c8110ba0 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.202+0000 7fb6d0000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8116860 con 0x7fb6c8075d30 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43526/0 (socket says 172.21.15.50:43526) 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 -- 172.21.15.50:0/1904807821 learned_addr learned my addr 172.21.15.50:0/1904807821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:02.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6ccc00700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 0x7fb6c819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:02.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6ce000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 0x7fb6c81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 -- 172.21.15.50:0/1904807821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 msgr2=0x7fb6c81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 0x7fb6c81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 -- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 msgr2=0x7fb6c819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 0x7fb6c819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.203+0000 7fb6cd600700 1 -- 172.21.15.50:0/1904807821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c81a1f20 con 0x7fb6c8075d30 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6cd600700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0047b0 tx=0x7fb6bc009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc027070 con 0x7fb6c8075d30 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6bc022470 con 0x7fb6c8075d30 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc017650 con 0x7fb6c8075d30 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81a21b0 con 0x7fb6c8075d30 2024-05-10T21:05:02.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.204+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81a9b30 con 0x7fb6c8075d30 2024-05-10T21:05:02.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.205+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c80512c0 con 0x7fb6c8075d30 2024-05-10T21:05:02.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.208+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb6bc02a070 con 0x7fb6c8075d30 2024-05-10T21:05:02.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.209+0000 7fb6c6000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 0x7fb6b00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:02.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.209+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb6bc0e3020 con 0x7fb6c8075d30 2024-05-10T21:05:02.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.210+0000 7fb6ccc00700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 0x7fb6b00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:02.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.210+0000 7fb6ccc00700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 0x7fb6b00b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8103500 tx=0x7fb6b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:02.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.214+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6bc013070 con 0x7fb6c8075d30 2024-05-10T21:05:02.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.490+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb6c8063c00 con 0x7fb6b00b2990 2024-05-10T21:05:02.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.499+0000 7fb6c6000700 1 -- 172.21.15.50:0/1904807821 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb6c8063c00 con 0x7fb6b00b2990 2024-05-10T21:05:02.503 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_0/ce1d7393-dde4-4f6e-b537-7fedef029d54 2024-05-10T21:05:02.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 msgr2=0x7fb6b00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:02.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 0x7fb6b00b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8103500 tx=0x7fb6b4009260 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 msgr2=0x7fb6c819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0047b0 tx=0x7fb6bc009550 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 shutdown_connections 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6b00b2990 0x7fb6b00b4ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c8110ba0 0x7fb6c81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c8075d30 0x7fb6c819d050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 --2- 172.21.15.50:0/1904807821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6c8076680 0x7fb6c819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 >> 172.21.15.50:0/1904807821 conn(0x7fb6c8100040 msgr2=0x7fb6c8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 shutdown_connections 2024-05-10T21:05:02.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:02.503+0000 7fb6d0000700 1 -- 172.21.15.50:0/1904807821 wait complete. 2024-05-10T21:05:02.877 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:02 smithi050 ceph-mon[29546]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:03.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:02 smithi092 ceph-mon[34415]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:03.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:02 smithi142 ceph-mon[34846]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:03.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T21:05:03.792 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:03.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:03 smithi050 ceph-mon[29546]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:04.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:03 smithi092 ceph-mon[34415]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:04.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:03 smithi142 ceph-mon[34846]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:04.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:04 smithi050 ceph-mon[29546]: pgmap v346: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:05.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:04 smithi092 ceph-mon[34415]: pgmap v346: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:05.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:04 smithi142 ceph-mon[34846]: pgmap v346: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 86 B/s, 1 objects/s recovering 2024-05-10T21:05:06.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.073+0000 7f07b3000700 1 -- 172.21.15.50:0/1907872370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac10a990 msgr2=0x7f07ac10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.073+0000 7f07b3000700 1 --2- 172.21.15.50:0/1907872370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac10a990 0x7f07ac10ada0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0798009990 tx=0x7f0798009ca0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 -- 172.21.15.50:0/1907872370 shutdown_connections 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 --2- 172.21.15.50:0/1907872370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 0x7f07ac117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 --2- 172.21.15.50:0/1907872370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac110970 0x7f07ac110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 --2- 172.21.15.50:0/1907872370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac10a990 0x7f07ac10ada0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 -- 172.21.15.50:0/1907872370 >> 172.21.15.50:0/1907872370 conn(0x7f07ac077c80 msgr2=0x7f07ac0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 -- 172.21.15.50:0/1907872370 shutdown_connections 2024-05-10T21:05:06.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.074+0000 7f07b3000700 1 -- 172.21.15.50:0/1907872370 wait complete. 2024-05-10T21:05:06.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.075+0000 7f07b3000700 1 Processor -- start 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.075+0000 7f07b3000700 1 -- start start 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 0x7f07ac1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 0x7f07ac1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac11aa90 con 0x7f07ac10b2e0 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac11a910 con 0x7f07ac110970 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b3000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac11ac10 con 0x7f07ac10a990 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07ab400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 0x7f07ac1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b1000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07abe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 0x7f07ac1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b1000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44808/0 (socket says 172.21.15.50:44808) 2024-05-10T21:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07b1000700 1 -- 172.21.15.50:0/2759083619 learned_addr learned my addr 172.21.15.50:0/2759083619 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.076+0000 7f07abe00700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 0x7f07ac1a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50822/0 (socket says 172.21.15.50:50822) 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 -- 172.21.15.50:0/2759083619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 msgr2=0x7f07ac1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 0x7f07ac1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 -- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 msgr2=0x7f07ac1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 0x7f07ac1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 -- 172.21.15.50:0/2759083619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac1a6310 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b1000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c370 tx=0x7f07a400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40092e0 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a4003a00 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac1a65a0 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.077+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40046f0 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.078+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac1a6ab0 con 0x7f07ac110970 2024-05-10T21:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.079+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac0512c0 con 0x7f07ac110970 2024-05-10T21:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.081+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f07a401a020 con 0x7f07ac110970 2024-05-10T21:05:06.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.083+0000 7f07a8c00700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 0x7f07900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:06.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.083+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f07a40e1bf0 con 0x7f07ac110970 2024-05-10T21:05:06.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.083+0000 7f07abe00700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 0x7f07900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:06.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.084+0000 7f07abe00700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 0x7f07900b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07ac07d0f0 tx=0x7f079c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:06.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.088+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07a4010040 con 0x7f07ac110970 2024-05-10T21:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.358+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f07ac063c00 con 0x7f07900b2990 2024-05-10T21:05:06.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.401+0000 7f07a8c00700 1 -- 172.21.15.50:0/2759083619 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07ac063c00 con 0x7f07900b2990 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 msgr2=0x7f07900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 0x7f07900b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07ac07d0f0 tx=0x7f079c009260 comp rx=0 tx=0).stop 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 msgr2=0x7f07ac1a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c370 tx=0x7f07a400c680 comp rx=0 tx=0).stop 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 shutdown_connections 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07900b2990 0x7f07900b4ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac110970 0x7f07ac1a5c70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07ac10a990 0x7f07ac1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 --2- 172.21.15.50:0/2759083619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07ac10b2e0 0x7f07ac1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:06.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.404+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 >> 172.21.15.50:0/2759083619 conn(0x7f07ac077c80 msgr2=0x7f07ac07b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:06.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.405+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 shutdown_connections 2024-05-10T21:05:06.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:06.405+0000 7f07b3000700 1 -- 172.21.15.50:0/2759083619 wait complete. 2024-05-10T21:05:06.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:06 smithi050 ceph-mon[29546]: pgmap v347: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s; 71 B/s, 1 objects/s recovering 2024-05-10T21:05:07.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:06 smithi092 ceph-mon[34415]: pgmap v347: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s; 71 B/s, 1 objects/s recovering 2024-05-10T21:05:07.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:06 smithi142 ceph-mon[34846]: pgmap v347: 97 pgs: 97 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s; 71 B/s, 1 objects/s recovering 2024-05-10T21:05:07.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T21:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:07.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:07 smithi050 ceph-mon[29546]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:08.034 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:07 smithi142 ceph-mon[34846]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:08.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:07 smithi092 ceph-mon[34415]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:08.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:08 smithi050 ceph-mon[29546]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 64 B/s, 1 objects/s recovering 2024-05-10T21:05:09.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:08 smithi092 ceph-mon[34415]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 64 B/s, 1 objects/s recovering 2024-05-10T21:05:09.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:08 smithi142 ceph-mon[34846]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 64 B/s, 1 objects/s recovering 2024-05-10T21:05:09.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 -- 172.21.15.50:0/2771440343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b0106ec0 msgr2=0x7f99b0113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:09.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 --2- 172.21.15.50:0/2771440343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b0106ec0 0x7f99b0113350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99a800b0a0 tx=0x7f99a800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 -- 172.21.15.50:0/2771440343 shutdown_connections 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 --2- 172.21.15.50:0/2771440343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010c5b0 0x7f99b010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 --2- 172.21.15.50:0/2771440343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b0106ec0 0x7f99b0113350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 --2- 172.21.15.50:0/2771440343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106570 0x7f99b0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 -- 172.21.15.50:0/2771440343 >> 172.21.15.50:0/2771440343 conn(0x7f99b0102050 msgr2=0x7f99b01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.982+0000 7f99b6200700 1 -- 172.21.15.50:0/2771440343 shutdown_connections 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.983+0000 7f99b6200700 1 -- 172.21.15.50:0/2771440343 wait complete. 2024-05-10T21:05:09.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.984+0000 7f99b6200700 1 Processor -- start 2024-05-10T21:05:09.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.984+0000 7f99b6200700 1 -- start start 2024-05-10T21:05:09.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:09.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 0x7f99b019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:09.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b010c5b0 0x7f99b01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01166e0 con 0x7f99b010c5b0 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0116560 con 0x7f99b0106570 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99b6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0116860 con 0x7f99b0106ec0 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.985+0000 7f99af400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99afe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b010c5b0 0x7f99b01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44824/0 (socket says 172.21.15.50:44824) 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 -- 172.21.15.50:0/1730050966 learned_addr learned my addr 172.21.15.50:0/1730050966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99a7e00700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 0x7f99b019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 -- 172.21.15.50:0/1730050966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 msgr2=0x7f99b019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 0x7f99b019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 -- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b010c5b0 msgr2=0x7f99b01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b010c5b0 0x7f99b01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 -- 172.21.15.50:0/1730050966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b01a1f60 con 0x7f99b0106570 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99a7e00700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 0x7f99b019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.986+0000 7f99af400700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e970 tx=0x7f99a000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.987+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004a30 con 0x7f99b0106570 2024-05-10T21:05:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.987+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b01a2250 con 0x7f99b0106570 2024-05-10T21:05:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.987+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a000cc90 con 0x7f99b0106570 2024-05-10T21:05:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.987+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0005610 con 0x7f99b0106570 2024-05-10T21:05:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.987+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a9b70 con 0x7f99b0106570 2024-05-10T21:05:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.988+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b00512c0 con 0x7f99b0106570 2024-05-10T21:05:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.991+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f99a000c7e0 con 0x7f99b0106570 2024-05-10T21:05:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.992+0000 7f99acc00700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 0x7f99940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.992+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f99a00e24f0 con 0x7f99b0106570 2024-05-10T21:05:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.992+0000 7f99a7e00700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 0x7f99940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:09.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.993+0000 7f99a7e00700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 0x7f99940b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99b01054b0 tx=0x7f9998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:10.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:09.997+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99a00aa080 con 0x7f99b0106570 2024-05-10T21:05:10.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.271+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f99b019e3f0 con 0x7f99940b2910 2024-05-10T21:05:10.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.279+0000 7f99acc00700 1 -- 172.21.15.50:0/1730050966 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f99b019e3f0 con 0x7f99940b2910 2024-05-10T21:05:10.283 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f 2024-05-10T21:05:10.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 msgr2=0x7f99940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 0x7f99940b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99b01054b0 tx=0x7f9998009260 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 msgr2=0x7f99b019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e970 tx=0x7f99a000ec80 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 shutdown_connections 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99940b2910 0x7f99940b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0106570 0x7f99b019d090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99b010c5b0 0x7f99b01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 --2- 172.21.15.50:0/1730050966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99b0106ec0 0x7f99b019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 >> 172.21.15.50:0/1730050966 conn(0x7f99b0102050 msgr2=0x7f99b01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 shutdown_connections 2024-05-10T21:05:10.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:10.282+0000 7f99b6200700 1 -- 172.21.15.50:0/1730050966 wait complete. 2024-05-10T21:05:10.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:10 smithi050 ceph-mon[29546]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 57 B/s, 1 objects/s recovering 2024-05-10T21:05:11.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:05:11.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:10 smithi092 ceph-mon[34415]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 57 B/s, 1 objects/s recovering 2024-05-10T21:05:11.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:10 smithi142 ceph-mon[34846]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 57 B/s, 1 objects/s recovering 2024-05-10T21:05:11.540 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:11.992 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:11 smithi050 ceph-mon[29546]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:12.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:11 smithi092 ceph-mon[34415]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:12.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:11 smithi142 ceph-mon[34846]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:05:13.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:12 smithi092 ceph-mon[34415]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:13.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:12 smithi050 ceph-mon[29546]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:13.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:12 smithi142 ceph-mon[34846]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:13.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 -- 172.21.15.50:0/3682506154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc10e910 msgr2=0x7f19bc110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:13.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 --2- 172.21.15.50:0/3682506154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc10e910 0x7f19bc110de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009f90 tx=0x7f19b400b330 comp rx=0 tx=0).stop 2024-05-10T21:05:13.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 -- 172.21.15.50:0/3682506154 shutdown_connections 2024-05-10T21:05:13.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 --2- 172.21.15.50:0/3682506154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc10e910 0x7f19bc110de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 --2- 172.21.15.50:0/3682506154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc1054e0 0x7f19bc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 --2- 172.21.15.50:0/3682506154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 -- 172.21.15.50:0/3682506154 >> 172.21.15.50:0/3682506154 conn(0x7f19bc0fedf0 msgr2=0x7f19bc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:13.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 -- 172.21.15.50:0/3682506154 shutdown_connections 2024-05-10T21:05:13.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.654+0000 7f19c3800700 1 -- 172.21.15.50:0/3682506154 wait complete. 2024-05-10T21:05:13.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.655+0000 7f19c3800700 1 Processor -- start 2024-05-10T21:05:13.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.655+0000 7f19c3800700 1 -- start start 2024-05-10T21:05:13.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.655+0000 7f19c3800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:13.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c3800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:13.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c3800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 0x7f19bc103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:13.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c3800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc1144b0 con 0x7f19bc10e910 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c3800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc114330 con 0x7f19bc1054e0 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c3800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc114630 con 0x7f19bc105e30 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19bbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44836/0 (socket says 172.21.15.50:44836) 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 -- 172.21.15.50:0/3579219818 learned_addr learned my addr 172.21.15.50:0/3579219818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c1800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 0x7f19bc103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 -- 172.21.15.50:0/3579219818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 msgr2=0x7f19bc103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc103710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 -- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 msgr2=0x7f19bc103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 0x7f19bc103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c0e00700 1 -- 172.21.15.50:0/3579219818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc1a9180 con 0x7f19bc1054e0 2024-05-10T21:05:13.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.656+0000 7f19c1800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 0x7f19bc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19bbe00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19c0e00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19b000e970 tx=0x7f19b000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b000cbd0 con 0x7f19bc1054e0 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b00044d0 con 0x7f19bc1054e0 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc1a9470 con 0x7f19bc1054e0 2024-05-10T21:05:13.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.657+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0003de0 con 0x7f19bc1054e0 2024-05-10T21:05:13.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.658+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc1a99b0 con 0x7f19bc1054e0 2024-05-10T21:05:13.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.659+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc0512c0 con 0x7f19bc1054e0 2024-05-10T21:05:13.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.661+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f19b0003f80 con 0x7f19bc1054e0 2024-05-10T21:05:13.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.663+0000 7f19b9600700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 0x7f19a40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:13.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.663+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f19b00e2660 con 0x7f19bc1054e0 2024-05-10T21:05:13.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.663+0000 7f19bbe00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 0x7f19a40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:13.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.664+0000 7f19bbe00700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 0x7f19a40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f19bc102380 tx=0x7f19a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:13.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.668+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19b00aa1f0 con 0x7f19bc1054e0 2024-05-10T21:05:13.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.982+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19bc02e5f0 con 0x7f19bc1054e0 2024-05-10T21:05:13.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:13 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:05:13.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.983+0000 7f19b9600700 1 -- 172.21.15.50:0/3579219818 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7f19b00a9af0 con 0x7f19bc1054e0 2024-05-10T21:05:13.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:13.987 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.986+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 msgr2=0x7f19a40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.986+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 0x7f19a40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f19bc102380 tx=0x7f19a8009310 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.986+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 msgr2=0x7f19bc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.986+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19b000e970 tx=0x7f19b000ed30 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 shutdown_connections 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19a40b2910 0x7f19a40b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19bc1054e0 0x7f19bc1031d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19bc10e910 0x7f19bc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 --2- 172.21.15.50:0/3579219818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc105e30 0x7f19bc103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:13.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 >> 172.21.15.50:0/3579219818 conn(0x7f19bc0fedf0 msgr2=0x7f19bc0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:13.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 shutdown_connections 2024-05-10T21:05:13.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:13.987+0000 7f19c3800700 1 -- 172.21.15.50:0/3579219818 wait complete. 2024-05-10T21:05:13.992 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:05:14.039 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:13 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:05:14.084 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:13 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:05:14.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:05:14.941 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:14 smithi050 ceph-mon[29546]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:14.941 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:14 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3579219818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:05:15.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:14 smithi092 ceph-mon[34415]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:15.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:14 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3579219818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:05:15.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:14 smithi142 ceph-mon[34846]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-10T21:05:15.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:14 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3579219818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:05:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:16.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:15 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:15 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:16.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:15 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:17.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:16 smithi092 ceph-mon[34415]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:05:17.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:16 smithi142 ceph-mon[34846]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:05:17.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:16 smithi050 ceph-mon[29546]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:05:18.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 -- 172.21.15.50:0/1141395169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c0764b0 msgr2=0x7f704c114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:18.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 --2- 172.21.15.50:0/1141395169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c0764b0 0x7f704c114e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f703c009990 tx=0x7f703c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 -- 172.21.15.50:0/1141395169 shutdown_connections 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 --2- 172.21.15.50:0/1141395169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c115370 0x7f704c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 --2- 172.21.15.50:0/1141395169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c075b60 0x7f704c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 --2- 172.21.15.50:0/1141395169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c0764b0 0x7f704c114e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 -- 172.21.15.50:0/1141395169 >> 172.21.15.50:0/1141395169 conn(0x7f704c100040 msgr2=0x7f704c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 -- 172.21.15.50:0/1141395169 shutdown_connections 2024-05-10T21:05:18.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.047+0000 7f7054800700 1 -- 172.21.15.50:0/1141395169 wait complete. 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.048+0000 7f7054800700 1 Processor -- start 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.048+0000 7f7054800700 1 -- start start 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c0764b0 0x7f704c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c115370 0x7f704c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f704c11aae0 con 0x7f704c115370 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f704c11a960 con 0x7f704c075b60 2024-05-10T21:05:18.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7054800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f704c11ac60 con 0x7f704c0764b0 2024-05-10T21:05:18.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7051e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:18.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7051e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:60600/0 (socket says 172.21.15.50:60600) 2024-05-10T21:05:18.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7051e00700 1 -- 172.21.15.50:0/1339281415 learned_addr learned my addr 172.21.15.50:0/1339281415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:18.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7052800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c115370 0x7f704c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:18.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.049+0000 7f7051400700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c0764b0 0x7f704c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 -- 172.21.15.50:0/1339281415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c0764b0 msgr2=0x7f704c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c0764b0 0x7f704c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 -- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c115370 msgr2=0x7f704c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c115370 0x7f704c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 -- 172.21.15.50:0/1339281415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a6380 con 0x7f704c075b60 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7051e00700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f704000e9c0 tx=0x7f704000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000cbd0 con 0x7f704c075b60 2024-05-10T21:05:18.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.050+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a6670 con 0x7f704c075b60 2024-05-10T21:05:18.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.051+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704000ee60 con 0x7f704c075b60 2024-05-10T21:05:18.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.051+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1adf90 con 0x7f704c075b60 2024-05-10T21:05:18.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.051+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400055c0 con 0x7f704c075b60 2024-05-10T21:05:18.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.052+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7030005330 con 0x7f704c075b60 2024-05-10T21:05:18.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.054+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7040004750 con 0x7f704c075b60 2024-05-10T21:05:18.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.056+0000 7f7046a00700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 0x7f70340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:18.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.056+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f70400e25a0 con 0x7f704c075b60 2024-05-10T21:05:18.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.056+0000 7f7051400700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 0x7f70340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:18.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.057+0000 7f7051400700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 0x7f70340b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f704c1034f0 tx=0x7f703c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:18.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.060+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70400aa130 con 0x7f704c075b60 2024-05-10T21:05:18.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.311+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70300058f0 con 0x7f704c075b60 2024-05-10T21:05:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.313+0000 7f7046a00700 1 -- 172.21.15.50:0/1339281415 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7f70400a9a30 con 0x7f704c075b60 2024-05-10T21:05:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:18.317 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:05:18.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 msgr2=0x7f70340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 0x7f70340b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f704c1034f0 tx=0x7f703c00b260 comp rx=0 tx=0).stop 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 msgr2=0x7f704c1a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f704000e9c0 tx=0x7f704000ecd0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 shutdown_connections 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70340b2910 0x7f70340b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f704c075b60 0x7f704c1a14b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f704c115370 0x7f704c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 --2- 172.21.15.50:0/1339281415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704c0764b0 0x7f704c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:18.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 >> 172.21.15.50:0/1339281415 conn(0x7f704c100040 msgr2=0x7f704c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:18.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 shutdown_connections 2024-05-10T21:05:18.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:18.316+0000 7f7054800700 1 -- 172.21.15.50:0/1339281415 wait complete. 2024-05-10T21:05:19.050 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:18 smithi050 ceph-mon[29546]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-10T21:05:19.051 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.051 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:18 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.051 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:18 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1339281415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:19.099 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T21:05:19.099 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:05:19.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:18 smithi092 ceph-mon[34415]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-10T21:05:19.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:18 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:18 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1339281415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:19.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:18 smithi142 ceph-mon[34846]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-10T21:05:19.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:18 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:19.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:18 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1339281415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:19.556 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:20.012 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:05:20.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:05:20.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:19 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:20.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:05:20.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:05:20.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:19 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:20.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:05:20.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:05:20.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:19 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:05:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:20 smithi092 ceph-mon[34415]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-10T21:05:21.132 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:20 smithi050 ceph-mon[29546]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-10T21:05:21.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:20 smithi142 ceph-mon[34846]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 -- 172.21.15.50:0/3983223722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228075d30 msgr2=0x7f3228076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 --2- 172.21.15.50:0/3983223722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228075d30 0x7f3228076140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f321c0098f0 tx=0x7f321c009c00 comp rx=0 tx=0).stop 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 -- 172.21.15.50:0/3983223722 shutdown_connections 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 --2- 172.21.15.50:0/3983223722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228110ba0 0x7f3228113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 --2- 172.21.15.50:0/3983223722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228075d30 0x7f3228076140 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:21.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 --2- 172.21.15.50:0/3983223722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f3228110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:21.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.735+0000 7f3230600700 1 -- 172.21.15.50:0/3983223722 >> 172.21.15.50:0/3983223722 conn(0x7f3228100040 msgr2=0x7f3228102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:21.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.736+0000 7f3230600700 1 -- 172.21.15.50:0/3983223722 shutdown_connections 2024-05-10T21:05:21.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.736+0000 7f3230600700 1 -- 172.21.15.50:0/3983223722 wait complete. 2024-05-10T21:05:21.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.737+0000 7f3230600700 1 Processor -- start 2024-05-10T21:05:21.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.737+0000 7f3230600700 1 -- start start 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f322819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228110ba0 0x7f32281a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3228116690 con 0x7f3228110ba0 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3228116510 con 0x7f3228075d30 2024-05-10T21:05:21.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.738+0000 7f3230600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3228116810 con 0x7f3228076680 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322e600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228110ba0 0x7f32281a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:60620/0 (socket says 172.21.15.50:60620) 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f322819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 -- 172.21.15.50:0/3275021907 learned_addr learned my addr 172.21.15.50:0/3275021907 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:21.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 -- 172.21.15.50:0/3275021907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 msgr2=0x7f322819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f322819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 -- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228110ba0 msgr2=0x7f32281a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228110ba0 0x7f32281a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322dc00700 1 -- 172.21.15.50:0/3275021907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32281a1f50 con 0x7f3228075d30 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.739+0000 7f322d200700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f322819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f322dc00700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f321c00f990 tx=0x7f321c00f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c027070 con 0x7f3228075d30 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281a21e0 con 0x7f3228075d30 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f321c0055f0 con 0x7f3228075d30 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c004930 con 0x7f3228075d30 2024-05-10T21:05:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.740+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281a9b60 con 0x7f3228075d30 2024-05-10T21:05:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.742+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32280772a0 con 0x7f3228075d30 2024-05-10T21:05:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.744+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f321c02a070 con 0x7f3228075d30 2024-05-10T21:05:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.746+0000 7f3226000700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 0x7f32100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.746+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f321c0e2ea0 con 0x7f3228075d30 2024-05-10T21:05:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.746+0000 7f322d200700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 0x7f32100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:21.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.747+0000 7f322d200700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 0x7f32100b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3214009750 tx=0x7f3214009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:21.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.748+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f321c013030 con 0x7f3228075d30 2024-05-10T21:05:21.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:21.980+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f322819e3e0 con 0x7f32100b2910 2024-05-10T21:05:22.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.007+0000 7f3226000700 1 -- 172.21.15.50:0/3275021907 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258741 (secure 0 0 0) 0x7f322819e3e0 con 0x7f32100b2910 2024-05-10T21:05:22.016 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:22.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 msgr2=0x7f32100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:22.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 0x7f32100b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3214009750 tx=0x7f3214009310 comp rx=0 tx=0).stop 2024-05-10T21:05:22.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 msgr2=0x7f322819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f321c00f990 tx=0x7f321c00f9c0 comp rx=0 tx=0).stop 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 shutdown_connections 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f32100b2910 0x7f32100b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3228075d30 0x7f322819d080 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228110ba0 0x7f32281a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 --2- 172.21.15.50:0/3275021907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3228076680 0x7f322819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 >> 172.21.15.50:0/3275021907 conn(0x7f3228100040 msgr2=0x7f3228101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 shutdown_connections 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:22.014+0000 7f3230600700 1 -- 172.21.15.50:0/3275021907 wait complete. 2024-05-10T21:05:22.019 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:05:22.766 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-10T21:05:21.586345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653418,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":291,"num_read_kb":234,"num_write":290,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374204,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567620,"statfs":{"total":1151940427776,"available":1151557242880,"internally_reserved":0,"allocated":12845056,"data_stored":7201827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13421,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":44,"num_write":12,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005619"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895293+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:04:56.895293+0000","last_peered":"2024-05-10T21:04:56.895293+0000","last_clean":"2024-05-10T21:04:56.895293+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:04:56.895293+0000","last_undegraded":"2024-05-10T21:04:56.895293+0000","last_fullsized":"2024-05-10T21:04:56.895293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889172+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:04:56.889172+0000","last_peered":"2024-05-10T21:04:56.889172+0000","last_clean":"2024-05-10T21:04:56.889172+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:04:56.889172+0000","last_undegraded":"2024-05-10T21:04:56.889172+0000","last_fullsized":"2024-05-10T21:04:56.889172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889027+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:04:56.889027+0000","last_peered":"2024-05-10T21:04:56.889027+0000","last_clean":"2024-05-10T21:04:56.889027+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:04:56.889027+0000","last_undegraded":"2024-05-10T21:04:56.889027+0000","last_fullsized":"2024-05-10T21:04:56.889027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331729+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:04:57.331729+0000","last_peered":"2024-05-10T21:04:57.331729+0000","last_clean":"2024-05-10T21:04:57.331729+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:04:57.331729+0000","last_undegraded":"2024-05-10T21:04:57.331729+0000","last_fullsized":"2024-05-10T21:04:57.331729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891141+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:04:56.891141+0000","last_peered":"2024-05-10T21:04:56.891141+0000","last_clean":"2024-05-10T21:04:56.891141+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:04:56.891141+0000","last_undegraded":"2024-05-10T21:04:56.891141+0000","last_fullsized":"2024-05-10T21:04:56.891141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886088+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:04:56.886088+0000","last_peered":"2024-05-10T21:04:56.886088+0000","last_clean":"2024-05-10T21:04:56.886088+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:04:56.886088+0000","last_undegraded":"2024-05-10T21:04:56.886088+0000","last_fullsized":"2024-05-10T21:04:56.886088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895454+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:04:56.895454+0000","last_peered":"2024-05-10T21:04:56.895454+0000","last_clean":"2024-05-10T21:04:56.895454+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:04:56.895454+0000","last_undegraded":"2024-05-10T21:04:56.895454+0000","last_fullsized":"2024-05-10T21:04:56.895454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.334654+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:04:57.334654+0000","last_peered":"2024-05-10T21:04:57.334654+0000","last_clean":"2024-05-10T21:04:57.334654+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:04:57.334654+0000","last_undegraded":"2024-05-10T21:04:57.334654+0000","last_fullsized":"2024-05-10T21:04:57.334654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.161010+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:04:57.161010+0000","last_peered":"2024-05-10T21:04:57.161010+0000","last_clean":"2024-05-10T21:04:57.161010+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:04:57.161010+0000","last_undegraded":"2024-05-10T21:04:57.161010+0000","last_fullsized":"2024-05-10T21:04:57.161010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896570+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:04:56.896570+0000","last_peered":"2024-05-10T21:04:56.896570+0000","last_clean":"2024-05-10T21:04:56.896570+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:04:56.896570+0000","last_undegraded":"2024-05-10T21:04:56.896570+0000","last_fullsized":"2024-05-10T21:04:56.896570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896323+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:04:56.896323+0000","last_peered":"2024-05-10T21:04:56.896323+0000","last_clean":"2024-05-10T21:04:56.896323+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:04:56.896323+0000","last_undegraded":"2024-05-10T21:04:56.896323+0000","last_fullsized":"2024-05-10T21:04:56.896323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331361+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:04:57.331361+0000","last_peered":"2024-05-10T21:04:57.331361+0000","last_clean":"2024-05-10T21:04:57.331361+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:04:57.331361+0000","last_undegraded":"2024-05-10T21:04:57.331361+0000","last_fullsized":"2024-05-10T21:04:57.331361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170963+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:04:57.170963+0000","last_peered":"2024-05-10T21:04:57.170963+0000","last_clean":"2024-05-10T21:04:57.170963+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:04:57.170963+0000","last_undegraded":"2024-05-10T21:04:57.170963+0000","last_fullsized":"2024-05-10T21:04:57.170963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888289+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:04:56.888289+0000","last_peered":"2024-05-10T21:04:56.888289+0000","last_clean":"2024-05-10T21:04:56.888289+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:04:56.888289+0000","last_undegraded":"2024-05-10T21:04:56.888289+0000","last_fullsized":"2024-05-10T21:04:56.888289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888016+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:04:56.888016+0000","last_peered":"2024-05-10T21:04:56.888016+0000","last_clean":"2024-05-10T21:04:56.888016+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:04:56.888016+0000","last_undegraded":"2024-05-10T21:04:56.888016+0000","last_fullsized":"2024-05-10T21:04:56.888016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160303+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:04:57.160303+0000","last_peered":"2024-05-10T21:04:57.160303+0000","last_clean":"2024-05-10T21:04:57.160303+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:04:57.160303+0000","last_undegraded":"2024-05-10T21:04:57.160303+0000","last_fullsized":"2024-05-10T21:04:57.160303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893696+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:04:56.893696+0000","last_peered":"2024-05-10T21:04:56.893696+0000","last_clean":"2024-05-10T21:04:56.893696+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:04:56.893696+0000","last_undegraded":"2024-05-10T21:04:56.893696+0000","last_fullsized":"2024-05-10T21:04:56.893696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.892489+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:04:56.892489+0000","last_peered":"2024-05-10T21:04:56.892489+0000","last_clean":"2024-05-10T21:04:56.892489+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:04:56.892489+0000","last_undegraded":"2024-05-10T21:04:56.892489+0000","last_fullsized":"2024-05-10T21:04:56.892489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894867+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:04:56.894867+0000","last_peered":"2024-05-10T21:04:56.894867+0000","last_clean":"2024-05-10T21:04:56.894867+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:04:56.894867+0000","last_undegraded":"2024-05-10T21:04:56.894867+0000","last_fullsized":"2024-05-10T21:04:56.894867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.183212+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.183212+0000","last_peered":"2024-05-10T21:04:57.183212+0000","last_clean":"2024-05-10T21:04:57.183212+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.183212+0000","last_undegraded":"2024-05-10T21:04:57.183212+0000","last_fullsized":"2024-05-10T21:04:57.183212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549238+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:04:57.549238+0000","last_peered":"2024-05-10T21:04:57.549238+0000","last_clean":"2024-05-10T21:04:57.549238+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:04:57.549238+0000","last_undegraded":"2024-05-10T21:04:57.549238+0000","last_fullsized":"2024-05-10T21:04:57.549238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549809+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:04:57.549809+0000","last_peered":"2024-05-10T21:04:57.549809+0000","last_clean":"2024-05-10T21:04:57.549809+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:04:57.549809+0000","last_undegraded":"2024-05-10T21:04:57.549809+0000","last_fullsized":"2024-05-10T21:04:57.549809+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890012+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:04:56.890012+0000","last_peered":"2024-05-10T21:04:56.890012+0000","last_clean":"2024-05-10T21:04:56.890012+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:04:56.890012+0000","last_undegraded":"2024-05-10T21:04:56.890012+0000","last_fullsized":"2024-05-10T21:04:56.890012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891580+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891580+0000","last_peered":"2024-05-10T21:04:56.891580+0000","last_clean":"2024-05-10T21:04:56.891580+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891580+0000","last_undegraded":"2024-05-10T21:04:56.891580+0000","last_fullsized":"2024-05-10T21:04:56.891580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160422+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:04:57.160422+0000","last_peered":"2024-05-10T21:04:57.160422+0000","last_clean":"2024-05-10T21:04:57.160422+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:04:57.160422+0000","last_undegraded":"2024-05-10T21:04:57.160422+0000","last_fullsized":"2024-05-10T21:04:57.160422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891333+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891333+0000","last_peered":"2024-05-10T21:04:56.891333+0000","last_clean":"2024-05-10T21:04:56.891333+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891333+0000","last_undegraded":"2024-05-10T21:04:56.891333+0000","last_fullsized":"2024-05-10T21:04:56.891333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330566+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:04:57.330566+0000","last_peered":"2024-05-10T21:04:57.330566+0000","last_clean":"2024-05-10T21:04:57.330566+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:04:57.330566+0000","last_undegraded":"2024-05-10T21:04:57.330566+0000","last_fullsized":"2024-05-10T21:04:57.330566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:18.826652+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:05:18.826652+0000","last_peered":"2024-05-10T21:05:18.826652+0000","last_clean":"2024-05-10T21:05:18.826652+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:05:18.826652+0000","last_undegraded":"2024-05-10T21:05:18.826652+0000","last_fullsized":"2024-05-10T21:05:18.826652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891071+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:04:56.891071+0000","last_peered":"2024-05-10T21:04:56.891071+0000","last_clean":"2024-05-10T21:04:56.891071+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:04:56.891071+0000","last_undegraded":"2024-05-10T21:04:56.891071+0000","last_fullsized":"2024-05-10T21:04:56.891071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.885924+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.885924+0000","last_peered":"2024-05-10T21:04:56.885924+0000","last_clean":"2024-05-10T21:04:56.885924+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.885924+0000","last_undegraded":"2024-05-10T21:04:56.885924+0000","last_fullsized":"2024-05-10T21:04:56.885924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.555691+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:04:57.555691+0000","last_peered":"2024-05-10T21:04:57.555691+0000","last_clean":"2024-05-10T21:04:57.555691+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:04:57.555691+0000","last_undegraded":"2024-05-10T21:04:57.555691+0000","last_fullsized":"2024-05-10T21:04:57.555691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170499+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170499+0000","last_peered":"2024-05-10T21:04:57.170499+0000","last_clean":"2024-05-10T21:04:57.170499+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170499+0000","last_undegraded":"2024-05-10T21:04:57.170499+0000","last_fullsized":"2024-05-10T21:04:57.170499+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548455+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:04:57.548455+0000","last_peered":"2024-05-10T21:04:57.548455+0000","last_clean":"2024-05-10T21:04:57.548455+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:04:57.548455+0000","last_undegraded":"2024-05-10T21:04:57.548455+0000","last_fullsized":"2024-05-10T21:04:57.548455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330788+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330788+0000","last_peered":"2024-05-10T21:04:57.330788+0000","last_clean":"2024-05-10T21:04:57.330788+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330788+0000","last_undegraded":"2024-05-10T21:04:57.330788+0000","last_fullsized":"2024-05-10T21:04:57.330788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890827+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:04:56.890827+0000","last_peered":"2024-05-10T21:04:56.890827+0000","last_clean":"2024-05-10T21:04:56.890827+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:04:56.890827+0000","last_undegraded":"2024-05-10T21:04:56.890827+0000","last_fullsized":"2024-05-10T21:04:56.890827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:15.821052+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:05:15.821052+0000","last_peered":"2024-05-10T21:05:15.821052+0000","last_clean":"2024-05-10T21:05:15.821052+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:05:15.821052+0000","last_undegraded":"2024-05-10T21:05:15.821052+0000","last_fullsized":"2024-05-10T21:05:15.821052+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160803+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:04:57.160803+0000","last_peered":"2024-05-10T21:04:57.160803+0000","last_clean":"2024-05-10T21:04:57.160803+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:04:57.160803+0000","last_undegraded":"2024-05-10T21:04:57.160803+0000","last_fullsized":"2024-05-10T21:04:57.160803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896142+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896142+0000","last_peered":"2024-05-10T21:04:56.896142+0000","last_clean":"2024-05-10T21:04:56.896142+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896142+0000","last_undegraded":"2024-05-10T21:04:56.896142+0000","last_fullsized":"2024-05-10T21:04:56.896142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889984+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:04:56.889984+0000","last_peered":"2024-05-10T21:04:56.889984+0000","last_clean":"2024-05-10T21:04:56.889984+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:04:56.889984+0000","last_undegraded":"2024-05-10T21:04:56.889984+0000","last_fullsized":"2024-05-10T21:04:56.889984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330748+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330748+0000","last_peered":"2024-05-10T21:04:57.330748+0000","last_clean":"2024-05-10T21:04:57.330748+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330748+0000","last_undegraded":"2024-05-10T21:04:57.330748+0000","last_fullsized":"2024-05-10T21:04:57.330748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160619+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:04:57.160619+0000","last_peered":"2024-05-10T21:04:57.160619+0000","last_clean":"2024-05-10T21:04:57.160619+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:04:57.160619+0000","last_undegraded":"2024-05-10T21:04:57.160619+0000","last_fullsized":"2024-05-10T21:04:57.160619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:16.822694+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:05:16.822694+0000","last_peered":"2024-05-10T21:05:16.822694+0000","last_clean":"2024-05-10T21:05:16.822694+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:05:16.822694+0000","last_undegraded":"2024-05-10T21:05:16.822694+0000","last_fullsized":"2024-05-10T21:05:16.822694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58815,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":71,"num_read_kb":92,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891341+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:04:56.891341+0000","last_peered":"2024-05-10T21:04:56.891341+0000","last_clean":"2024-05-10T21:04:56.891341+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:04:56.891341+0000","last_undegraded":"2024-05-10T21:04:56.891341+0000","last_fullsized":"2024-05-10T21:04:56.891341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894610+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894610+0000","last_peered":"2024-05-10T21:04:56.894610+0000","last_clean":"2024-05-10T21:04:56.894610+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894610+0000","last_undegraded":"2024-05-10T21:04:56.894610+0000","last_fullsized":"2024-05-10T21:04:56.894610+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170216+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:04:57.170216+0000","last_peered":"2024-05-10T21:04:57.170216+0000","last_clean":"2024-05-10T21:04:57.170216+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:04:57.170216+0000","last_undegraded":"2024-05-10T21:04:57.170216+0000","last_fullsized":"2024-05-10T21:04:57.170216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330390+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330390+0000","last_peered":"2024-05-10T21:04:57.330390+0000","last_clean":"2024-05-10T21:04:57.330390+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330390+0000","last_undegraded":"2024-05-10T21:04:57.330390+0000","last_fullsized":"2024-05-10T21:04:57.330390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896300+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:04:56.896300+0000","last_peered":"2024-05-10T21:04:56.896300+0000","last_clean":"2024-05-10T21:04:56.896300+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:04:56.896300+0000","last_undegraded":"2024-05-10T21:04:56.896300+0000","last_fullsized":"2024-05-10T21:04:56.896300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:16.836598+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:05:16.836598+0000","last_peered":"2024-05-10T21:05:16.836598+0000","last_clean":"2024-05-10T21:05:16.836598+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:05:16.836598+0000","last_undegraded":"2024-05-10T21:05:16.836598+0000","last_fullsized":"2024-05-10T21:05:16.836598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895387+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:04:56.895387+0000","last_peered":"2024-05-10T21:04:56.895387+0000","last_clean":"2024-05-10T21:04:56.895387+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:04:56.895387+0000","last_undegraded":"2024-05-10T21:04:56.895387+0000","last_fullsized":"2024-05-10T21:04:56.895387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.353561+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.353561+0000","last_peered":"2024-05-10T21:04:57.353561+0000","last_clean":"2024-05-10T21:04:57.353561+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.353561+0000","last_undegraded":"2024-05-10T21:04:57.353561+0000","last_fullsized":"2024-05-10T21:04:57.353561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330390+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:04:57.330390+0000","last_peered":"2024-05-10T21:04:57.330390+0000","last_clean":"2024-05-10T21:04:57.330390+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:04:57.330390+0000","last_undegraded":"2024-05-10T21:04:57.330390+0000","last_fullsized":"2024-05-10T21:04:57.330390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891679+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891679+0000","last_peered":"2024-05-10T21:04:56.891679+0000","last_clean":"2024-05-10T21:04:56.891679+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891679+0000","last_undegraded":"2024-05-10T21:04:56.891679+0000","last_fullsized":"2024-05-10T21:04:56.891679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894322+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:04:56.894322+0000","last_peered":"2024-05-10T21:04:56.894322+0000","last_clean":"2024-05-10T21:04:56.894322+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:04:56.894322+0000","last_undegraded":"2024-05-10T21:04:56.894322+0000","last_fullsized":"2024-05-10T21:04:56.894322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.892904+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.892904+0000","last_peered":"2024-05-10T21:04:56.892904+0000","last_clean":"2024-05-10T21:04:56.892904+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.892904+0000","last_undegraded":"2024-05-10T21:04:56.892904+0000","last_fullsized":"2024-05-10T21:04:56.892904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886362+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:04:56.886362+0000","last_peered":"2024-05-10T21:04:56.886362+0000","last_clean":"2024-05-10T21:04:56.886362+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:04:56.886362+0000","last_undegraded":"2024-05-10T21:04:56.886362+0000","last_fullsized":"2024-05-10T21:04:56.886362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895608+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895608+0000","last_peered":"2024-05-10T21:04:56.895608+0000","last_clean":"2024-05-10T21:04:56.895608+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895608+0000","last_undegraded":"2024-05-10T21:04:56.895608+0000","last_fullsized":"2024-05-10T21:04:56.895608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886078+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:04:56.886078+0000","last_peered":"2024-05-10T21:04:56.886078+0000","last_clean":"2024-05-10T21:04:56.886078+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:04:56.886078+0000","last_undegraded":"2024-05-10T21:04:56.886078+0000","last_fullsized":"2024-05-10T21:04:56.886078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331003+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:04:57.331003+0000","last_peered":"2024-05-10T21:04:57.331003+0000","last_clean":"2024-05-10T21:04:57.331003+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:04:57.331003+0000","last_undegraded":"2024-05-10T21:04:57.331003+0000","last_fullsized":"2024-05-10T21:04:57.331003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:14.832855+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:05:14.832855+0000","last_peered":"2024-05-10T21:05:14.832855+0000","last_clean":"2024-05-10T21:05:14.832855+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:05:14.832855+0000","last_undegraded":"2024-05-10T21:05:14.832855+0000","last_fullsized":"2024-05-10T21:05:14.832855+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549039+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:04:57.549039+0000","last_peered":"2024-05-10T21:04:57.549039+0000","last_clean":"2024-05-10T21:04:57.549039+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:04:57.549039+0000","last_undegraded":"2024-05-10T21:04:57.549039+0000","last_fullsized":"2024-05-10T21:04:57.549039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331796+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331796+0000","last_peered":"2024-05-10T21:04:57.331796+0000","last_clean":"2024-05-10T21:04:57.331796+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331796+0000","last_undegraded":"2024-05-10T21:04:57.331796+0000","last_fullsized":"2024-05-10T21:04:57.331796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895557+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:04:56.895557+0000","last_peered":"2024-05-10T21:04:56.895557+0000","last_clean":"2024-05-10T21:04:56.895557+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:04:56.895557+0000","last_undegraded":"2024-05-10T21:04:56.895557+0000","last_fullsized":"2024-05-10T21:04:56.895557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.169693+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:04:57.169693+0000","last_peered":"2024-05-10T21:04:57.169693+0000","last_clean":"2024-05-10T21:04:57.169693+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:04:57.169693+0000","last_undegraded":"2024-05-10T21:04:57.169693+0000","last_fullsized":"2024-05-10T21:04:57.169693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895614+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:04:56.895614+0000","last_peered":"2024-05-10T21:04:56.895614+0000","last_clean":"2024-05-10T21:04:56.895614+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:04:56.895614+0000","last_undegraded":"2024-05-10T21:04:56.895614+0000","last_fullsized":"2024-05-10T21:04:56.895614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891495+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891495+0000","last_peered":"2024-05-10T21:04:56.891495+0000","last_clean":"2024-05-10T21:04:56.891495+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891495+0000","last_undegraded":"2024-05-10T21:04:56.891495+0000","last_fullsized":"2024-05-10T21:04:56.891495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170194+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:04:57.170194+0000","last_peered":"2024-05-10T21:04:57.170194+0000","last_clean":"2024-05-10T21:04:57.170194+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:04:57.170194+0000","last_undegraded":"2024-05-10T21:04:57.170194+0000","last_fullsized":"2024-05-10T21:04:57.170194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:11.957412+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:05:11.957412+0000","last_peered":"2024-05-10T21:05:11.957412+0000","last_clean":"2024-05-10T21:05:11.957412+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:05:11.957412+0000","last_undegraded":"2024-05-10T21:05:11.957412+0000","last_fullsized":"2024-05-10T21:05:11.957412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549431+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:04:57.549431+0000","last_peered":"2024-05-10T21:04:57.549431+0000","last_clean":"2024-05-10T21:04:57.549431+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:04:57.549431+0000","last_undegraded":"2024-05-10T21:04:57.549431+0000","last_fullsized":"2024-05-10T21:04:57.549431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331299+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331299+0000","last_peered":"2024-05-10T21:04:57.331299+0000","last_clean":"2024-05-10T21:04:57.331299+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331299+0000","last_undegraded":"2024-05-10T21:04:57.331299+0000","last_fullsized":"2024-05-10T21:04:57.331299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889542+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:04:56.889542+0000","last_peered":"2024-05-10T21:04:56.889542+0000","last_clean":"2024-05-10T21:04:56.889542+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:04:56.889542+0000","last_undegraded":"2024-05-10T21:04:56.889542+0000","last_fullsized":"2024-05-10T21:04:56.889542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160338+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160338+0000","last_peered":"2024-05-10T21:04:57.160338+0000","last_clean":"2024-05-10T21:04:57.160338+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160338+0000","last_undegraded":"2024-05-10T21:04:57.160338+0000","last_fullsized":"2024-05-10T21:04:57.160338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890999+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:04:56.890999+0000","last_peered":"2024-05-10T21:04:56.890999+0000","last_clean":"2024-05-10T21:04:56.890999+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:04:56.890999+0000","last_undegraded":"2024-05-10T21:04:56.890999+0000","last_fullsized":"2024-05-10T21:04:56.890999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548646+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548646+0000","last_peered":"2024-05-10T21:04:57.548646+0000","last_clean":"2024-05-10T21:04:57.548646+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548646+0000","last_undegraded":"2024-05-10T21:04:57.548646+0000","last_fullsized":"2024-05-10T21:04:57.548646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893321+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:04:56.893321+0000","last_peered":"2024-05-10T21:04:56.893321+0000","last_clean":"2024-05-10T21:04:56.893321+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:04:56.893321+0000","last_undegraded":"2024-05-10T21:04:56.893321+0000","last_fullsized":"2024-05-10T21:04:56.893321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894257+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894257+0000","last_peered":"2024-05-10T21:04:56.894257+0000","last_clean":"2024-05-10T21:04:56.894257+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894257+0000","last_undegraded":"2024-05-10T21:04:56.894257+0000","last_fullsized":"2024-05-10T21:04:56.894257+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.550246+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:04:57.550246+0000","last_peered":"2024-05-10T21:04:57.550246+0000","last_clean":"2024-05-10T21:04:57.550246+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:04:57.550246+0000","last_undegraded":"2024-05-10T21:04:57.550246+0000","last_fullsized":"2024-05-10T21:04:57.550246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.340707+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.340707+0000","last_peered":"2024-05-10T21:04:57.340707+0000","last_clean":"2024-05-10T21:04:57.340707+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.340707+0000","last_undegraded":"2024-05-10T21:04:57.340707+0000","last_fullsized":"2024-05-10T21:04:57.340707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895787+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:04:56.895787+0000","last_peered":"2024-05-10T21:04:56.895787+0000","last_clean":"2024-05-10T21:04:56.895787+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:04:56.895787+0000","last_undegraded":"2024-05-10T21:04:56.895787+0000","last_fullsized":"2024-05-10T21:04:56.895787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895025+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895025+0000","last_peered":"2024-05-10T21:04:56.895025+0000","last_clean":"2024-05-10T21:04:56.895025+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895025+0000","last_undegraded":"2024-05-10T21:04:56.895025+0000","last_fullsized":"2024-05-10T21:04:56.895025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896188+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:04:56.896188+0000","last_peered":"2024-05-10T21:04:56.896188+0000","last_clean":"2024-05-10T21:04:56.896188+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:04:56.896188+0000","last_undegraded":"2024-05-10T21:04:56.896188+0000","last_fullsized":"2024-05-10T21:04:56.896188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896014+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:04:56.896014+0000","last_peered":"2024-05-10T21:04:56.896014+0000","last_clean":"2024-05-10T21:04:56.896014+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:04:56.896014+0000","last_undegraded":"2024-05-10T21:04:56.896014+0000","last_fullsized":"2024-05-10T21:04:56.896014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171577+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171577+0000","last_peered":"2024-05-10T21:04:57.171577+0000","last_clean":"2024-05-10T21:04:57.171577+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171577+0000","last_undegraded":"2024-05-10T21:04:57.171577+0000","last_fullsized":"2024-05-10T21:04:57.171577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896271+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:04:56.896271+0000","last_peered":"2024-05-10T21:04:56.896271+0000","last_clean":"2024-05-10T21:04:56.896271+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:04:56.896271+0000","last_undegraded":"2024-05-10T21:04:56.896271+0000","last_fullsized":"2024-05-10T21:04:56.896271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893550+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:04:56.893550+0000","last_peered":"2024-05-10T21:04:56.893550+0000","last_clean":"2024-05-10T21:04:56.893550+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:04:56.893550+0000","last_undegraded":"2024-05-10T21:04:56.893550+0000","last_fullsized":"2024-05-10T21:04:56.893550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894126+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:04:56.894126+0000","last_peered":"2024-05-10T21:04:56.894126+0000","last_clean":"2024-05-10T21:04:56.894126+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:04:56.894126+0000","last_undegraded":"2024-05-10T21:04:56.894126+0000","last_fullsized":"2024-05-10T21:04:56.894126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894084+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:04:56.894084+0000","last_peered":"2024-05-10T21:04:56.894084+0000","last_clean":"2024-05-10T21:04:56.894084+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:04:56.894084+0000","last_undegraded":"2024-05-10T21:04:56.894084+0000","last_fullsized":"2024-05-10T21:04:56.894084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888782+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:04:56.888782+0000","last_peered":"2024-05-10T21:04:56.888782+0000","last_clean":"2024-05-10T21:04:56.888782+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:04:56.888782+0000","last_undegraded":"2024-05-10T21:04:56.888782+0000","last_fullsized":"2024-05-10T21:04:56.888782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171188+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:04:57.171188+0000","last_peered":"2024-05-10T21:04:57.171188+0000","last_clean":"2024-05-10T21:04:57.171188+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:04:57.171188+0000","last_undegraded":"2024-05-10T21:04:57.171188+0000","last_fullsized":"2024-05-10T21:04:57.171188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894100+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:04:56.894100+0000","last_peered":"2024-05-10T21:04:56.894100+0000","last_clean":"2024-05-10T21:04:56.894100+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:04:56.894100+0000","last_undegraded":"2024-05-10T21:04:56.894100+0000","last_fullsized":"2024-05-10T21:04:56.894100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895039+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:04:56.895039+0000","last_peered":"2024-05-10T21:04:56.895039+0000","last_clean":"2024-05-10T21:04:56.895039+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:04:56.895039+0000","last_undegraded":"2024-05-10T21:04:56.895039+0000","last_fullsized":"2024-05-10T21:04:56.895039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549254+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:04:57.549254+0000","last_peered":"2024-05-10T21:04:57.549254+0000","last_clean":"2024-05-10T21:04:57.549254+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:04:57.549254+0000","last_undegraded":"2024-05-10T21:04:57.549254+0000","last_fullsized":"2024-05-10T21:04:57.549254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330800+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:04:57.330800+0000","last_peered":"2024-05-10T21:04:57.330800+0000","last_clean":"2024-05-10T21:04:57.330800+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:04:57.330800+0000","last_undegraded":"2024-05-10T21:04:57.330800+0000","last_fullsized":"2024-05-10T21:04:57.330800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548467+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:04:57.548467+0000","last_peered":"2024-05-10T21:04:57.548467+0000","last_clean":"2024-05-10T21:04:57.548467+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:04:57.548467+0000","last_undegraded":"2024-05-10T21:04:57.548467+0000","last_fullsized":"2024-05-10T21:04:57.548467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160684+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:04:57.160684+0000","last_peered":"2024-05-10T21:04:57.160684+0000","last_clean":"2024-05-10T21:04:57.160684+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:04:57.160684+0000","last_undegraded":"2024-05-10T21:04:57.160684+0000","last_fullsized":"2024-05-10T21:04:57.160684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170619+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:04:57.170619+0000","last_peered":"2024-05-10T21:04:57.170619+0000","last_clean":"2024-05-10T21:04:57.170619+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:04:57.170619+0000","last_undegraded":"2024-05-10T21:04:57.170619+0000","last_fullsized":"2024-05-10T21:04:57.170619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893691+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:04:56.893691+0000","last_peered":"2024-05-10T21:04:56.893691+0000","last_clean":"2024-05-10T21:04:56.893691+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:04:56.893691+0000","last_undegraded":"2024-05-10T21:04:56.893691+0000","last_fullsized":"2024-05-10T21:04:56.893691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":215,"num_read_kb":170,"num_write":165,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":192705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1585152,"data_stored":1087712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626488,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":495840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":500753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":437590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:05:22.773 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:05:23.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:22 smithi092 ceph-mon[34415]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-10T21:05:23.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:22 smithi142 ceph-mon[34846]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-10T21:05:23.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:22 smithi050 ceph-mon[29546]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-10T21:05:23.285 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:24.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:23 smithi092 ceph-mon[34415]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:24.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:23 smithi142 ceph-mon[34846]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:24.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:23 smithi050 ceph-mon[29546]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:25.044 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:24 smithi050 ceph-mon[29546]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-10T21:05:25.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:24 smithi092 ceph-mon[34415]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-10T21:05:25.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:24 smithi142 ceph-mon[34846]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 -- 172.21.15.50:0/1421754882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae8105be0 msgr2=0x7f3ae810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 --2- 172.21.15.50:0/1421754882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae8105be0 0x7f3ae810e130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40098f0 tx=0x7f3ad4009c00 comp rx=0 tx=0).stop 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 -- 172.21.15.50:0/1421754882 shutdown_connections 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 --2- 172.21.15.50:0/1421754882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105290 0x7f3ae81056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 --2- 172.21.15.50:0/1421754882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae810e700 0x7f3ae8110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 --2- 172.21.15.50:0/1421754882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae8105be0 0x7f3ae810e130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.615+0000 7f3aee000700 1 -- 172.21.15.50:0/1421754882 >> 172.21.15.50:0/1421754882 conn(0x7f3ae8078ac0 msgr2=0x7f3ae8078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.616+0000 7f3aee000700 1 -- 172.21.15.50:0/1421754882 shutdown_connections 2024-05-10T21:05:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.616+0000 7f3aee000700 1 -- 172.21.15.50:0/1421754882 wait complete. 2024-05-10T21:05:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.617+0000 7f3aee000700 1 Processor -- start 2024-05-10T21:05:25.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 -- start start 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 0x7f3ae8102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105be0 0x7f3ae81034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 0x7f3ae8103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8114260 con 0x7f3ae8105290 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81140e0 con 0x7f3ae8105be0 2024-05-10T21:05:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.618+0000 7f3aee000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81143e0 con 0x7f3ae810e700 2024-05-10T21:05:25.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aecc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 0x7f3ae8102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:25.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 0x7f3ae8103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3ae7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105be0 0x7f3ae81034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aecc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 0x7f3ae8102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55152/0 (socket says 172.21.15.50:55152) 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aecc00700 1 -- 172.21.15.50:0/4118755175 learned_addr learned my addr 172.21.15.50:0/4118755175 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 -- 172.21.15.50:0/4118755175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105be0 msgr2=0x7f3ae81034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105be0 0x7f3ae81034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 -- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 msgr2=0x7f3ae8102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 0x7f3ae8102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.619+0000 7f3aed600700 1 -- 172.21.15.50:0/4118755175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae81a9070 con 0x7f3ae810e700 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3aed600700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 0x7f3ae8103a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000efc0 tx=0x7f3ae000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00092e0 con 0x7f3ae810e700 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ae0004bb0 con 0x7f3ae810e700 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004020 con 0x7f3ae810e700 2024-05-10T21:05:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae81a9300 con 0x7f3ae810e700 2024-05-10T21:05:25.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.620+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81a97c0 con 0x7f3ae810e700 2024-05-10T21:05:25.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.621+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae806bee0 con 0x7f3ae810e700 2024-05-10T21:05:25.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.624+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3ae001b030 con 0x7f3ae810e700 2024-05-10T21:05:25.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.625+0000 7f3ae5600700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 0x7f3ad00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:25.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.626+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f3ae00e2700 con 0x7f3ae810e700 2024-05-10T21:05:25.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.626+0000 7f3aecc00700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 0x7f3ad00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:25.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.626+0000 7f3aecc00700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 0x7f3ad00b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d20 tx=0x7f3adc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:25.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.628+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ae0022020 con 0x7f3ae810e700 2024-05-10T21:05:25.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.862+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ae8063c00 con 0x7f3ad00b2990 2024-05-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.880+0000 7f3ae5600700 1 -- 172.21.15.50:0/4118755175 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258707 (secure 0 0 0) 0x7f3ae8063c00 con 0x7f3ad00b2990 2024-05-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 msgr2=0x7f3ad00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 0x7f3ad00b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d20 tx=0x7f3adc009260 comp rx=0 tx=0).stop 2024-05-10T21:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 msgr2=0x7f3ae8103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 0x7f3ae8103a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000efc0 tx=0x7f3ae000c3a0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 shutdown_connections 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3ad00b2990 0x7f3ad00b4ea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ae810e700 0x7f3ae8103a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ae8105290 0x7f3ae8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 --2- 172.21.15.50:0/4118755175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae8105be0 0x7f3ae81034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 >> 172.21.15.50:0/4118755175 conn(0x7f3ae8078ac0 msgr2=0x7f3ae81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 shutdown_connections 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:25.888+0000 7f3aee000700 1 -- 172.21.15.50:0/4118755175 wait complete. 2024-05-10T21:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:05:26.953 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-05-10T21:05:25.588793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657675,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":317,"num_read_kb":254,"num_write":291,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374212,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567612,"statfs":{"total":1151940427776,"available":1151557234688,"internally_reserved":0,"allocated":12853248,"data_stored":7210341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":40,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006400"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895293+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:04:56.895293+0000","last_peered":"2024-05-10T21:04:56.895293+0000","last_clean":"2024-05-10T21:04:56.895293+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:04:56.895293+0000","last_undegraded":"2024-05-10T21:04:56.895293+0000","last_fullsized":"2024-05-10T21:04:56.895293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889172+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:04:56.889172+0000","last_peered":"2024-05-10T21:04:56.889172+0000","last_clean":"2024-05-10T21:04:56.889172+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:04:56.889172+0000","last_undegraded":"2024-05-10T21:04:56.889172+0000","last_fullsized":"2024-05-10T21:04:56.889172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889027+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:04:56.889027+0000","last_peered":"2024-05-10T21:04:56.889027+0000","last_clean":"2024-05-10T21:04:56.889027+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:04:56.889027+0000","last_undegraded":"2024-05-10T21:04:56.889027+0000","last_fullsized":"2024-05-10T21:04:56.889027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331729+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:04:57.331729+0000","last_peered":"2024-05-10T21:04:57.331729+0000","last_clean":"2024-05-10T21:04:57.331729+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:04:57.331729+0000","last_undegraded":"2024-05-10T21:04:57.331729+0000","last_fullsized":"2024-05-10T21:04:57.331729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891141+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:04:56.891141+0000","last_peered":"2024-05-10T21:04:56.891141+0000","last_clean":"2024-05-10T21:04:56.891141+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:04:56.891141+0000","last_undegraded":"2024-05-10T21:04:56.891141+0000","last_fullsized":"2024-05-10T21:04:56.891141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886088+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:04:56.886088+0000","last_peered":"2024-05-10T21:04:56.886088+0000","last_clean":"2024-05-10T21:04:56.886088+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:04:56.886088+0000","last_undegraded":"2024-05-10T21:04:56.886088+0000","last_fullsized":"2024-05-10T21:04:56.886088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895454+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:04:56.895454+0000","last_peered":"2024-05-10T21:04:56.895454+0000","last_clean":"2024-05-10T21:04:56.895454+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:04:56.895454+0000","last_undegraded":"2024-05-10T21:04:56.895454+0000","last_fullsized":"2024-05-10T21:04:56.895454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.334654+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:04:57.334654+0000","last_peered":"2024-05-10T21:04:57.334654+0000","last_clean":"2024-05-10T21:04:57.334654+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:04:57.334654+0000","last_undegraded":"2024-05-10T21:04:57.334654+0000","last_fullsized":"2024-05-10T21:04:57.334654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.161010+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:04:57.161010+0000","last_peered":"2024-05-10T21:04:57.161010+0000","last_clean":"2024-05-10T21:04:57.161010+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:04:57.161010+0000","last_undegraded":"2024-05-10T21:04:57.161010+0000","last_fullsized":"2024-05-10T21:04:57.161010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896570+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:04:56.896570+0000","last_peered":"2024-05-10T21:04:56.896570+0000","last_clean":"2024-05-10T21:04:56.896570+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:04:56.896570+0000","last_undegraded":"2024-05-10T21:04:56.896570+0000","last_fullsized":"2024-05-10T21:04:56.896570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896323+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:04:56.896323+0000","last_peered":"2024-05-10T21:04:56.896323+0000","last_clean":"2024-05-10T21:04:56.896323+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:04:56.896323+0000","last_undegraded":"2024-05-10T21:04:56.896323+0000","last_fullsized":"2024-05-10T21:04:56.896323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331361+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:04:57.331361+0000","last_peered":"2024-05-10T21:04:57.331361+0000","last_clean":"2024-05-10T21:04:57.331361+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:04:57.331361+0000","last_undegraded":"2024-05-10T21:04:57.331361+0000","last_fullsized":"2024-05-10T21:04:57.331361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170963+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:04:57.170963+0000","last_peered":"2024-05-10T21:04:57.170963+0000","last_clean":"2024-05-10T21:04:57.170963+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:04:57.170963+0000","last_undegraded":"2024-05-10T21:04:57.170963+0000","last_fullsized":"2024-05-10T21:04:57.170963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888289+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:04:56.888289+0000","last_peered":"2024-05-10T21:04:56.888289+0000","last_clean":"2024-05-10T21:04:56.888289+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:04:56.888289+0000","last_undegraded":"2024-05-10T21:04:56.888289+0000","last_fullsized":"2024-05-10T21:04:56.888289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888016+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:04:56.888016+0000","last_peered":"2024-05-10T21:04:56.888016+0000","last_clean":"2024-05-10T21:04:56.888016+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:04:56.888016+0000","last_undegraded":"2024-05-10T21:04:56.888016+0000","last_fullsized":"2024-05-10T21:04:56.888016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160303+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:04:57.160303+0000","last_peered":"2024-05-10T21:04:57.160303+0000","last_clean":"2024-05-10T21:04:57.160303+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:04:57.160303+0000","last_undegraded":"2024-05-10T21:04:57.160303+0000","last_fullsized":"2024-05-10T21:04:57.160303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893696+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:04:56.893696+0000","last_peered":"2024-05-10T21:04:56.893696+0000","last_clean":"2024-05-10T21:04:56.893696+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:04:56.893696+0000","last_undegraded":"2024-05-10T21:04:56.893696+0000","last_fullsized":"2024-05-10T21:04:56.893696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.892489+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:04:56.892489+0000","last_peered":"2024-05-10T21:04:56.892489+0000","last_clean":"2024-05-10T21:04:56.892489+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:04:56.892489+0000","last_undegraded":"2024-05-10T21:04:56.892489+0000","last_fullsized":"2024-05-10T21:04:56.892489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894867+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:04:56.894867+0000","last_peered":"2024-05-10T21:04:56.894867+0000","last_clean":"2024-05-10T21:04:56.894867+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:04:56.894867+0000","last_undegraded":"2024-05-10T21:04:56.894867+0000","last_fullsized":"2024-05-10T21:04:56.894867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.183212+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.183212+0000","last_peered":"2024-05-10T21:04:57.183212+0000","last_clean":"2024-05-10T21:04:57.183212+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.183212+0000","last_undegraded":"2024-05-10T21:04:57.183212+0000","last_fullsized":"2024-05-10T21:04:57.183212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549238+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:04:57.549238+0000","last_peered":"2024-05-10T21:04:57.549238+0000","last_clean":"2024-05-10T21:04:57.549238+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:04:57.549238+0000","last_undegraded":"2024-05-10T21:04:57.549238+0000","last_fullsized":"2024-05-10T21:04:57.549238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549809+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:04:57.549809+0000","last_peered":"2024-05-10T21:04:57.549809+0000","last_clean":"2024-05-10T21:04:57.549809+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:04:57.549809+0000","last_undegraded":"2024-05-10T21:04:57.549809+0000","last_fullsized":"2024-05-10T21:04:57.549809+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890012+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:04:56.890012+0000","last_peered":"2024-05-10T21:04:56.890012+0000","last_clean":"2024-05-10T21:04:56.890012+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:04:56.890012+0000","last_undegraded":"2024-05-10T21:04:56.890012+0000","last_fullsized":"2024-05-10T21:04:56.890012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891580+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891580+0000","last_peered":"2024-05-10T21:04:56.891580+0000","last_clean":"2024-05-10T21:04:56.891580+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891580+0000","last_undegraded":"2024-05-10T21:04:56.891580+0000","last_fullsized":"2024-05-10T21:04:56.891580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160422+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:04:57.160422+0000","last_peered":"2024-05-10T21:04:57.160422+0000","last_clean":"2024-05-10T21:04:57.160422+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:04:57.160422+0000","last_undegraded":"2024-05-10T21:04:57.160422+0000","last_fullsized":"2024-05-10T21:04:57.160422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891333+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891333+0000","last_peered":"2024-05-10T21:04:56.891333+0000","last_clean":"2024-05-10T21:04:56.891333+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891333+0000","last_undegraded":"2024-05-10T21:04:56.891333+0000","last_fullsized":"2024-05-10T21:04:56.891333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330566+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:04:57.330566+0000","last_peered":"2024-05-10T21:04:57.330566+0000","last_clean":"2024-05-10T21:04:57.330566+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:04:57.330566+0000","last_undegraded":"2024-05-10T21:04:57.330566+0000","last_fullsized":"2024-05-10T21:04:57.330566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:18.826652+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:05:18.826652+0000","last_peered":"2024-05-10T21:05:18.826652+0000","last_clean":"2024-05-10T21:05:18.826652+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:05:18.826652+0000","last_undegraded":"2024-05-10T21:05:18.826652+0000","last_fullsized":"2024-05-10T21:05:18.826652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891071+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:04:56.891071+0000","last_peered":"2024-05-10T21:04:56.891071+0000","last_clean":"2024-05-10T21:04:56.891071+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:04:56.891071+0000","last_undegraded":"2024-05-10T21:04:56.891071+0000","last_fullsized":"2024-05-10T21:04:56.891071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.885924+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.885924+0000","last_peered":"2024-05-10T21:04:56.885924+0000","last_clean":"2024-05-10T21:04:56.885924+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.885924+0000","last_undegraded":"2024-05-10T21:04:56.885924+0000","last_fullsized":"2024-05-10T21:04:56.885924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.555691+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:04:57.555691+0000","last_peered":"2024-05-10T21:04:57.555691+0000","last_clean":"2024-05-10T21:04:57.555691+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:04:57.555691+0000","last_undegraded":"2024-05-10T21:04:57.555691+0000","last_fullsized":"2024-05-10T21:04:57.555691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170499+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170499+0000","last_peered":"2024-05-10T21:04:57.170499+0000","last_clean":"2024-05-10T21:04:57.170499+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170499+0000","last_undegraded":"2024-05-10T21:04:57.170499+0000","last_fullsized":"2024-05-10T21:04:57.170499+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548455+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:04:57.548455+0000","last_peered":"2024-05-10T21:04:57.548455+0000","last_clean":"2024-05-10T21:04:57.548455+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:04:57.548455+0000","last_undegraded":"2024-05-10T21:04:57.548455+0000","last_fullsized":"2024-05-10T21:04:57.548455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330788+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330788+0000","last_peered":"2024-05-10T21:04:57.330788+0000","last_clean":"2024-05-10T21:04:57.330788+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330788+0000","last_undegraded":"2024-05-10T21:04:57.330788+0000","last_fullsized":"2024-05-10T21:04:57.330788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890827+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:04:56.890827+0000","last_peered":"2024-05-10T21:04:56.890827+0000","last_clean":"2024-05-10T21:04:56.890827+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:04:56.890827+0000","last_undegraded":"2024-05-10T21:04:56.890827+0000","last_fullsized":"2024-05-10T21:04:56.890827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:20.829120+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:05:20.829120+0000","last_peered":"2024-05-10T21:05:20.829120+0000","last_clean":"2024-05-10T21:05:20.829120+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:05:20.829120+0000","last_undegraded":"2024-05-10T21:05:20.829120+0000","last_fullsized":"2024-05-10T21:05:20.829120+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160803+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:04:57.160803+0000","last_peered":"2024-05-10T21:04:57.160803+0000","last_clean":"2024-05-10T21:04:57.160803+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:04:57.160803+0000","last_undegraded":"2024-05-10T21:04:57.160803+0000","last_fullsized":"2024-05-10T21:04:57.160803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896142+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896142+0000","last_peered":"2024-05-10T21:04:56.896142+0000","last_clean":"2024-05-10T21:04:56.896142+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896142+0000","last_undegraded":"2024-05-10T21:04:56.896142+0000","last_fullsized":"2024-05-10T21:04:56.896142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889984+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:04:56.889984+0000","last_peered":"2024-05-10T21:04:56.889984+0000","last_clean":"2024-05-10T21:04:56.889984+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:04:56.889984+0000","last_undegraded":"2024-05-10T21:04:56.889984+0000","last_fullsized":"2024-05-10T21:04:56.889984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330748+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330748+0000","last_peered":"2024-05-10T21:04:57.330748+0000","last_clean":"2024-05-10T21:04:57.330748+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330748+0000","last_undegraded":"2024-05-10T21:04:57.330748+0000","last_fullsized":"2024-05-10T21:04:57.330748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160619+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:04:57.160619+0000","last_peered":"2024-05-10T21:04:57.160619+0000","last_clean":"2024-05-10T21:04:57.160619+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:04:57.160619+0000","last_undegraded":"2024-05-10T21:04:57.160619+0000","last_fullsized":"2024-05-10T21:04:57.160619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'30","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:21.830921+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:05:21.830921+0000","last_peered":"2024-05-10T21:05:21.830921+0000","last_clean":"2024-05-10T21:05:21.830921+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:05:21.830921+0000","last_undegraded":"2024-05-10T21:05:21.830921+0000","last_fullsized":"2024-05-10T21:05:21.830921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":102,"num_write":31,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891341+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:04:56.891341+0000","last_peered":"2024-05-10T21:04:56.891341+0000","last_clean":"2024-05-10T21:04:56.891341+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:04:56.891341+0000","last_undegraded":"2024-05-10T21:04:56.891341+0000","last_fullsized":"2024-05-10T21:04:56.891341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894610+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894610+0000","last_peered":"2024-05-10T21:04:56.894610+0000","last_clean":"2024-05-10T21:04:56.894610+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894610+0000","last_undegraded":"2024-05-10T21:04:56.894610+0000","last_fullsized":"2024-05-10T21:04:56.894610+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170216+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:04:57.170216+0000","last_peered":"2024-05-10T21:04:57.170216+0000","last_clean":"2024-05-10T21:04:57.170216+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:04:57.170216+0000","last_undegraded":"2024-05-10T21:04:57.170216+0000","last_fullsized":"2024-05-10T21:04:57.170216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330390+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330390+0000","last_peered":"2024-05-10T21:04:57.330390+0000","last_clean":"2024-05-10T21:04:57.330390+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330390+0000","last_undegraded":"2024-05-10T21:04:57.330390+0000","last_fullsized":"2024-05-10T21:04:57.330390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896300+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:04:56.896300+0000","last_peered":"2024-05-10T21:04:56.896300+0000","last_clean":"2024-05-10T21:04:56.896300+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:04:56.896300+0000","last_undegraded":"2024-05-10T21:04:56.896300+0000","last_fullsized":"2024-05-10T21:04:56.896300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:21.846207+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:05:21.846207+0000","last_peered":"2024-05-10T21:05:21.846207+0000","last_clean":"2024-05-10T21:05:21.846207+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:05:21.846207+0000","last_undegraded":"2024-05-10T21:05:21.846207+0000","last_fullsized":"2024-05-10T21:05:21.846207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895387+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:04:56.895387+0000","last_peered":"2024-05-10T21:04:56.895387+0000","last_clean":"2024-05-10T21:04:56.895387+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:04:56.895387+0000","last_undegraded":"2024-05-10T21:04:56.895387+0000","last_fullsized":"2024-05-10T21:04:56.895387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.353561+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.353561+0000","last_peered":"2024-05-10T21:04:57.353561+0000","last_clean":"2024-05-10T21:04:57.353561+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.353561+0000","last_undegraded":"2024-05-10T21:04:57.353561+0000","last_fullsized":"2024-05-10T21:04:57.353561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330390+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:04:57.330390+0000","last_peered":"2024-05-10T21:04:57.330390+0000","last_clean":"2024-05-10T21:04:57.330390+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:04:57.330390+0000","last_undegraded":"2024-05-10T21:04:57.330390+0000","last_fullsized":"2024-05-10T21:04:57.330390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891679+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891679+0000","last_peered":"2024-05-10T21:04:56.891679+0000","last_clean":"2024-05-10T21:04:56.891679+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891679+0000","last_undegraded":"2024-05-10T21:04:56.891679+0000","last_fullsized":"2024-05-10T21:04:56.891679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894322+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:04:56.894322+0000","last_peered":"2024-05-10T21:04:56.894322+0000","last_clean":"2024-05-10T21:04:56.894322+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:04:56.894322+0000","last_undegraded":"2024-05-10T21:04:56.894322+0000","last_fullsized":"2024-05-10T21:04:56.894322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.892904+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.892904+0000","last_peered":"2024-05-10T21:04:56.892904+0000","last_clean":"2024-05-10T21:04:56.892904+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.892904+0000","last_undegraded":"2024-05-10T21:04:56.892904+0000","last_fullsized":"2024-05-10T21:04:56.892904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886362+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:04:56.886362+0000","last_peered":"2024-05-10T21:04:56.886362+0000","last_clean":"2024-05-10T21:04:56.886362+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:04:56.886362+0000","last_undegraded":"2024-05-10T21:04:56.886362+0000","last_fullsized":"2024-05-10T21:04:56.886362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895608+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895608+0000","last_peered":"2024-05-10T21:04:56.895608+0000","last_clean":"2024-05-10T21:04:56.895608+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895608+0000","last_undegraded":"2024-05-10T21:04:56.895608+0000","last_fullsized":"2024-05-10T21:04:56.895608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886078+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:04:56.886078+0000","last_peered":"2024-05-10T21:04:56.886078+0000","last_clean":"2024-05-10T21:04:56.886078+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:04:56.886078+0000","last_undegraded":"2024-05-10T21:04:56.886078+0000","last_fullsized":"2024-05-10T21:04:56.886078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331003+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:04:57.331003+0000","last_peered":"2024-05-10T21:04:57.331003+0000","last_clean":"2024-05-10T21:04:57.331003+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:04:57.331003+0000","last_undegraded":"2024-05-10T21:04:57.331003+0000","last_fullsized":"2024-05-10T21:04:57.331003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.841549+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:05:19.841549+0000","last_peered":"2024-05-10T21:05:19.841549+0000","last_clean":"2024-05-10T21:05:19.841549+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:05:19.841549+0000","last_undegraded":"2024-05-10T21:05:19.841549+0000","last_fullsized":"2024-05-10T21:05:19.841549+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549039+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:04:57.549039+0000","last_peered":"2024-05-10T21:04:57.549039+0000","last_clean":"2024-05-10T21:04:57.549039+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:04:57.549039+0000","last_undegraded":"2024-05-10T21:04:57.549039+0000","last_fullsized":"2024-05-10T21:04:57.549039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331796+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331796+0000","last_peered":"2024-05-10T21:04:57.331796+0000","last_clean":"2024-05-10T21:04:57.331796+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331796+0000","last_undegraded":"2024-05-10T21:04:57.331796+0000","last_fullsized":"2024-05-10T21:04:57.331796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895557+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:04:56.895557+0000","last_peered":"2024-05-10T21:04:56.895557+0000","last_clean":"2024-05-10T21:04:56.895557+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:04:56.895557+0000","last_undegraded":"2024-05-10T21:04:56.895557+0000","last_fullsized":"2024-05-10T21:04:56.895557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.169693+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:04:57.169693+0000","last_peered":"2024-05-10T21:04:57.169693+0000","last_clean":"2024-05-10T21:04:57.169693+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:04:57.169693+0000","last_undegraded":"2024-05-10T21:04:57.169693+0000","last_fullsized":"2024-05-10T21:04:57.169693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895614+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:04:56.895614+0000","last_peered":"2024-05-10T21:04:56.895614+0000","last_clean":"2024-05-10T21:04:56.895614+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:04:56.895614+0000","last_undegraded":"2024-05-10T21:04:56.895614+0000","last_fullsized":"2024-05-10T21:04:56.895614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891495+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891495+0000","last_peered":"2024-05-10T21:04:56.891495+0000","last_clean":"2024-05-10T21:04:56.891495+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891495+0000","last_undegraded":"2024-05-10T21:04:56.891495+0000","last_fullsized":"2024-05-10T21:04:56.891495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170194+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:04:57.170194+0000","last_peered":"2024-05-10T21:04:57.170194+0000","last_clean":"2024-05-10T21:04:57.170194+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:04:57.170194+0000","last_undegraded":"2024-05-10T21:04:57.170194+0000","last_fullsized":"2024-05-10T21:04:57.170194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:11.957412+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:05:11.957412+0000","last_peered":"2024-05-10T21:05:11.957412+0000","last_clean":"2024-05-10T21:05:11.957412+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:05:11.957412+0000","last_undegraded":"2024-05-10T21:05:11.957412+0000","last_fullsized":"2024-05-10T21:05:11.957412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549431+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:04:57.549431+0000","last_peered":"2024-05-10T21:04:57.549431+0000","last_clean":"2024-05-10T21:04:57.549431+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:04:57.549431+0000","last_undegraded":"2024-05-10T21:04:57.549431+0000","last_fullsized":"2024-05-10T21:04:57.549431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331299+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331299+0000","last_peered":"2024-05-10T21:04:57.331299+0000","last_clean":"2024-05-10T21:04:57.331299+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331299+0000","last_undegraded":"2024-05-10T21:04:57.331299+0000","last_fullsized":"2024-05-10T21:04:57.331299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.889542+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:04:56.889542+0000","last_peered":"2024-05-10T21:04:56.889542+0000","last_clean":"2024-05-10T21:04:56.889542+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:04:56.889542+0000","last_undegraded":"2024-05-10T21:04:56.889542+0000","last_fullsized":"2024-05-10T21:04:56.889542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160338+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160338+0000","last_peered":"2024-05-10T21:04:57.160338+0000","last_clean":"2024-05-10T21:04:57.160338+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160338+0000","last_undegraded":"2024-05-10T21:04:57.160338+0000","last_fullsized":"2024-05-10T21:04:57.160338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.890999+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:04:56.890999+0000","last_peered":"2024-05-10T21:04:56.890999+0000","last_clean":"2024-05-10T21:04:56.890999+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:04:56.890999+0000","last_undegraded":"2024-05-10T21:04:56.890999+0000","last_fullsized":"2024-05-10T21:04:56.890999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548646+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548646+0000","last_peered":"2024-05-10T21:04:57.548646+0000","last_clean":"2024-05-10T21:04:57.548646+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548646+0000","last_undegraded":"2024-05-10T21:04:57.548646+0000","last_fullsized":"2024-05-10T21:04:57.548646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893321+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:04:56.893321+0000","last_peered":"2024-05-10T21:04:56.893321+0000","last_clean":"2024-05-10T21:04:56.893321+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:04:56.893321+0000","last_undegraded":"2024-05-10T21:04:56.893321+0000","last_fullsized":"2024-05-10T21:04:56.893321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894257+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894257+0000","last_peered":"2024-05-10T21:04:56.894257+0000","last_clean":"2024-05-10T21:04:56.894257+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894257+0000","last_undegraded":"2024-05-10T21:04:56.894257+0000","last_fullsized":"2024-05-10T21:04:56.894257+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.550246+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:04:57.550246+0000","last_peered":"2024-05-10T21:04:57.550246+0000","last_clean":"2024-05-10T21:04:57.550246+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:04:57.550246+0000","last_undegraded":"2024-05-10T21:04:57.550246+0000","last_fullsized":"2024-05-10T21:04:57.550246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.340707+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.340707+0000","last_peered":"2024-05-10T21:04:57.340707+0000","last_clean":"2024-05-10T21:04:57.340707+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.340707+0000","last_undegraded":"2024-05-10T21:04:57.340707+0000","last_fullsized":"2024-05-10T21:04:57.340707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895787+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:04:56.895787+0000","last_peered":"2024-05-10T21:04:56.895787+0000","last_clean":"2024-05-10T21:04:56.895787+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:04:56.895787+0000","last_undegraded":"2024-05-10T21:04:56.895787+0000","last_fullsized":"2024-05-10T21:04:56.895787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895025+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895025+0000","last_peered":"2024-05-10T21:04:56.895025+0000","last_clean":"2024-05-10T21:04:56.895025+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895025+0000","last_undegraded":"2024-05-10T21:04:56.895025+0000","last_fullsized":"2024-05-10T21:04:56.895025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896188+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:04:56.896188+0000","last_peered":"2024-05-10T21:04:56.896188+0000","last_clean":"2024-05-10T21:04:56.896188+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:04:56.896188+0000","last_undegraded":"2024-05-10T21:04:56.896188+0000","last_fullsized":"2024-05-10T21:04:56.896188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896014+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:04:56.896014+0000","last_peered":"2024-05-10T21:04:56.896014+0000","last_clean":"2024-05-10T21:04:56.896014+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:04:56.896014+0000","last_undegraded":"2024-05-10T21:04:56.896014+0000","last_fullsized":"2024-05-10T21:04:56.896014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171577+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171577+0000","last_peered":"2024-05-10T21:04:57.171577+0000","last_clean":"2024-05-10T21:04:57.171577+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171577+0000","last_undegraded":"2024-05-10T21:04:57.171577+0000","last_fullsized":"2024-05-10T21:04:57.171577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896271+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:04:56.896271+0000","last_peered":"2024-05-10T21:04:56.896271+0000","last_clean":"2024-05-10T21:04:56.896271+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:04:56.896271+0000","last_undegraded":"2024-05-10T21:04:56.896271+0000","last_fullsized":"2024-05-10T21:04:56.896271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893550+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:04:56.893550+0000","last_peered":"2024-05-10T21:04:56.893550+0000","last_clean":"2024-05-10T21:04:56.893550+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:04:56.893550+0000","last_undegraded":"2024-05-10T21:04:56.893550+0000","last_fullsized":"2024-05-10T21:04:56.893550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894126+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:04:56.894126+0000","last_peered":"2024-05-10T21:04:56.894126+0000","last_clean":"2024-05-10T21:04:56.894126+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:04:56.894126+0000","last_undegraded":"2024-05-10T21:04:56.894126+0000","last_fullsized":"2024-05-10T21:04:56.894126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894084+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:04:56.894084+0000","last_peered":"2024-05-10T21:04:56.894084+0000","last_clean":"2024-05-10T21:04:56.894084+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:04:56.894084+0000","last_undegraded":"2024-05-10T21:04:56.894084+0000","last_fullsized":"2024-05-10T21:04:56.894084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.888782+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:04:56.888782+0000","last_peered":"2024-05-10T21:04:56.888782+0000","last_clean":"2024-05-10T21:04:56.888782+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:04:56.888782+0000","last_undegraded":"2024-05-10T21:04:56.888782+0000","last_fullsized":"2024-05-10T21:04:56.888782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171188+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:04:57.171188+0000","last_peered":"2024-05-10T21:04:57.171188+0000","last_clean":"2024-05-10T21:04:57.171188+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:04:57.171188+0000","last_undegraded":"2024-05-10T21:04:57.171188+0000","last_fullsized":"2024-05-10T21:04:57.171188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894100+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:04:56.894100+0000","last_peered":"2024-05-10T21:04:56.894100+0000","last_clean":"2024-05-10T21:04:56.894100+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:04:56.894100+0000","last_undegraded":"2024-05-10T21:04:56.894100+0000","last_fullsized":"2024-05-10T21:04:56.894100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895039+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:04:56.895039+0000","last_peered":"2024-05-10T21:04:56.895039+0000","last_clean":"2024-05-10T21:04:56.895039+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:04:56.895039+0000","last_undegraded":"2024-05-10T21:04:56.895039+0000","last_fullsized":"2024-05-10T21:04:56.895039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549254+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:04:57.549254+0000","last_peered":"2024-05-10T21:04:57.549254+0000","last_clean":"2024-05-10T21:04:57.549254+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:04:57.549254+0000","last_undegraded":"2024-05-10T21:04:57.549254+0000","last_fullsized":"2024-05-10T21:04:57.549254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330800+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:04:57.330800+0000","last_peered":"2024-05-10T21:04:57.330800+0000","last_clean":"2024-05-10T21:04:57.330800+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:04:57.330800+0000","last_undegraded":"2024-05-10T21:04:57.330800+0000","last_fullsized":"2024-05-10T21:04:57.330800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548467+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:04:57.548467+0000","last_peered":"2024-05-10T21:04:57.548467+0000","last_clean":"2024-05-10T21:04:57.548467+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:04:57.548467+0000","last_undegraded":"2024-05-10T21:04:57.548467+0000","last_fullsized":"2024-05-10T21:04:57.548467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160684+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:04:57.160684+0000","last_peered":"2024-05-10T21:04:57.160684+0000","last_clean":"2024-05-10T21:04:57.160684+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:04:57.160684+0000","last_undegraded":"2024-05-10T21:04:57.160684+0000","last_fullsized":"2024-05-10T21:04:57.160684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170619+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:04:57.170619+0000","last_peered":"2024-05-10T21:04:57.170619+0000","last_clean":"2024-05-10T21:04:57.170619+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:04:57.170619+0000","last_undegraded":"2024-05-10T21:04:57.170619+0000","last_fullsized":"2024-05-10T21:04:57.170619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893691+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:04:56.893691+0000","last_peered":"2024-05-10T21:04:56.893691+0000","last_clean":"2024-05-10T21:04:56.893691+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:04:56.893691+0000","last_undegraded":"2024-05-10T21:04:56.893691+0000","last_fullsized":"2024-05-10T21:04:56.893691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":241,"num_read_kb":190,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841570,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1091969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234031,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":500097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":500753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":437590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:05:26.958 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T21:05:26.959 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T21:05:26.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T21:05:27.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:26 smithi092 ceph-mon[34415]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:05:27.122 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:26 smithi092 ceph-mon[34415]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:27.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:26 smithi142 ceph-mon[34846]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:05:27.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:26 smithi142 ceph-mon[34846]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:27.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:26 smithi050 ceph-mon[29546]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:05:27.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:26 smithi050 ceph-mon[29546]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:05:27.424 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:29.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:28 smithi050 ceph-mon[29546]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:05:29.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:28 smithi092 ceph-mon[34415]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:05:29.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:28 smithi142 ceph-mon[34846]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.698+0000 7fe9fda00700 1 -- 172.21.15.50:0/3841920608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8107680 msgr2=0x7fe9f8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.698+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3841920608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8107680 0x7fe9f8107a90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40098f0 tx=0x7fe9e4009c00 comp rx=0 tx=0).stop 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 -- 172.21.15.50:0/3841920608 shutdown_connections 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3841920608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8108020 0x7fe9f810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3841920608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8106550 0x7fe9f8106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:29.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3841920608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8107680 0x7fe9f8107a90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 -- 172.21.15.50:0/3841920608 >> 172.21.15.50:0/3841920608 conn(0x7fe9f8101b90 msgr2=0x7fe9f8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 -- 172.21.15.50:0/3841920608 shutdown_connections 2024-05-10T21:05:29.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.699+0000 7fe9fda00700 1 -- 172.21.15.50:0/3841920608 wait complete. 2024-05-10T21:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.700+0000 7fe9fda00700 1 Processor -- start 2024-05-10T21:05:29.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.700+0000 7fe9fda00700 1 -- start start 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.700+0000 7fe9fda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 0x7fe9f819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.700+0000 7fe9fda00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 0x7fe9f819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9fda00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 0x7fe9f81a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9fda00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8111fb0 con 0x7fe9f8107680 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9fda00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8111e30 con 0x7fe9f8108020 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9fda00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8112130 con 0x7fe9f8106550 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f6000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 0x7fe9f819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 0x7fe9f819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f6000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 0x7fe9f819d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55172/0 (socket says 172.21.15.50:55172) 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 0x7fe9f81a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:29.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f6000700 1 -- 172.21.15.50:0/3187806539 learned_addr learned my addr 172.21.15.50:0/3187806539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 0x7fe9f819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35072/0 (socket says 172.21.15.50:35072) 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f7400700 1 -- 172.21.15.50:0/3187806539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 msgr2=0x7fe9f819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f7400700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 0x7fe9f819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f7400700 1 -- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 msgr2=0x7fe9f819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.701+0000 7fe9f7400700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 0x7fe9f819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9f7400700 1 -- 172.21.15.50:0/3187806539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a4660 con 0x7fe9f8108020 2024-05-10T21:05:29.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9f7400700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 0x7fe9f81a3fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000efc0 tx=0x7fe9f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00092e0 con 0x7fe9f8108020 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f0004bb0 con 0x7fe9f8108020 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0004020 con 0x7fe9f8108020 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a48f0 con 0x7fe9f8108020 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.702+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81a4e30 con 0x7fe9f8108020 2024-05-10T21:05:29.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.704+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f80512c0 con 0x7fe9f8108020 2024-05-10T21:05:29.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.706+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe9f001b030 con 0x7fe9f8108020 2024-05-10T21:05:29.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.708+0000 7fe9ef400700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 0x7fe9dc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.708+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fe9f00e1e20 con 0x7fe9f8108020 2024-05-10T21:05:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.708+0000 7fe9f6a00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 0x7fe9dc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:29.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.709+0000 7fe9f6a00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 0x7fe9dc0b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8107580 tx=0x7fe9e8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:29.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:29.712+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9f0022020 con 0x7fe9f8108020 2024-05-10T21:05:30.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.044+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe9f80643d0 con 0x7fe9f8108020 2024-05-10T21:05:30.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.053+0000 7fe9ef400700 1 -- 172.21.15.50:0/3187806539 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe9f0010040 con 0x7fe9f8108020 2024-05-10T21:05:30.056 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-05-10T21:05:30.057 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAajD5mpc7JAhAAmbL4gGaiZZmv5yqiY24uWw== 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 msgr2=0x7fe9dc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 0x7fe9dc0b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8107580 tx=0x7fe9e8009310 comp rx=0 tx=0).stop 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 msgr2=0x7fe9f81a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 0x7fe9f81a3fe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000efc0 tx=0x7fe9f000c3a0 comp rx=0 tx=0).stop 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 shutdown_connections 2024-05-10T21:05:30.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9dc0b2990 0x7fe9dc0b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f8108020 0x7fe9f81a3fe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9f8106550 0x7fe9f819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 --2- 172.21.15.50:0/3187806539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9f8107680 0x7fe9f819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 >> 172.21.15.50:0/3187806539 conn(0x7fe9f8101b90 msgr2=0x7fe9f810a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.057+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 shutdown_connections 2024-05-10T21:05:30.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:30.058+0000 7fe9fda00700 1 -- 172.21.15.50:0/3187806539 wait complete. 2024-05-10T21:05:30.953 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:05:30.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T21:05:30.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T21:05:31.100 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T21:05:31.100 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T21:05:31.101 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T21:05:31.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:30 smithi092 ceph-mon[34415]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:31.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:30 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3187806539' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:30 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:30 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T21:05:31.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:30 smithi142 ceph-mon[34846]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:31.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:30 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3187806539' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:30 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:30 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T21:05:31.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:30 smithi050 ceph-mon[29546]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:31.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:30 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3187806539' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:30 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T21:05:31.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:30 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T21:05:31.600 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:32.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:32 smithi050 ceph-mon[29546]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:33.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:32 smithi092 ceph-mon[34415]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:33.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:32 smithi142 ceph-mon[34846]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.050+0000 7f25eb200700 1 -- 172.21.15.50:0/1332798157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 msgr2=0x7f25ec109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.050+0000 7f25eb200700 1 --2- 172.21.15.50:0/1332798157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec109140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25d40098f0 tx=0x7f25d4009c00 comp rx=0 tx=0).stop 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 -- 172.21.15.50:0/1332798157 shutdown_connections 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 --2- 172.21.15.50:0/1332798157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 0x7f25ec110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 --2- 172.21.15.50:0/1332798157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 --2- 172.21.15.50:0/1332798157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec109140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 -- 172.21.15.50:0/1332798157 >> 172.21.15.50:0/1332798157 conn(0x7f25ec079d10 msgr2=0x7f25ec07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 -- 172.21.15.50:0/1332798157 shutdown_connections 2024-05-10T21:05:34.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.051+0000 7f25eb200700 1 -- 172.21.15.50:0/1332798157 wait complete. 2024-05-10T21:05:34.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.052+0000 7f25eb200700 1 Processor -- start 2024-05-10T21:05:34.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.052+0000 7f25eb200700 1 -- start start 2024-05-10T21:05:34.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:34.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 0x7f25ec076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1142e0 con 0x7f25ec104400 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec114160 con 0x7f25ec108d30 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25eb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec114460 con 0x7f25ec109680 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25e9e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52324/0 (socket says 172.21.15.50:52324) 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25e9e00700 1 -- 172.21.15.50:0/1184754336 learned_addr learned my addr 172.21.15.50:0/1184754336 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25e9400700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.053+0000 7f25ea800700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 0x7f25ec076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 -- 172.21.15.50:0/1184754336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 msgr2=0x7f25ec076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 0x7f25ec076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 -- 172.21.15.50:0/1184754336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 msgr2=0x7f25ec076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 -- 172.21.15.50:0/1184754336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec1a90c0 con 0x7f25ec104400 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9400700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e9e00700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9c0 tx=0x7f25dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc004a30 con 0x7f25ec104400 2024-05-10T21:05:34.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.054+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25dc00cc70 con 0x7f25ec104400 2024-05-10T21:05:34.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.055+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec1a93b0 con 0x7f25ec104400 2024-05-10T21:05:34.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.055+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc005490 con 0x7f25ec104400 2024-05-10T21:05:34.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.055+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec1a98c0 con 0x7f25ec104400 2024-05-10T21:05:34.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.056+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0512c0 con 0x7f25ec104400 2024-05-10T21:05:34.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.058+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f25dc004bd0 con 0x7f25ec104400 2024-05-10T21:05:34.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.060+0000 7f25e2a00700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 0x7f25d00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:34.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.060+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f25dc0e24a0 con 0x7f25ec104400 2024-05-10T21:05:34.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.060+0000 7f25e9400700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 0x7f25d00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:34.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.061+0000 7f25e9400700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 0x7f25d00b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009fd0 tx=0x7f25d4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:34.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.065+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25dc0aa030 con 0x7f25ec104400 2024-05-10T21:05:34.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.372+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f25ec02e5f0 con 0x7f25ec104400 2024-05-10T21:05:34.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.392+0000 7f25e2a00700 1 -- 172.21.15.50:0/1184754336 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288642 (secure 0 0 0) 0x7f25dc0a9930 con 0x7f25ec104400 2024-05-10T21:05:34.396 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 msgr2=0x7f25d00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 0x7f25d00b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009fd0 tx=0x7f25d4019040 comp rx=0 tx=0).stop 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 msgr2=0x7f25ec075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9c0 tx=0x7f25dc00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 shutdown_connections 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f25d00b2910 0x7f25d00b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25ec109680 0x7f25ec076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25ec104400 0x7f25ec075ac0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 --2- 172.21.15.50:0/1184754336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec108d30 0x7f25ec076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:34.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 >> 172.21.15.50:0/1184754336 conn(0x7f25ec079d10 msgr2=0x7f25ec101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:34.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 shutdown_connections 2024-05-10T21:05:34.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:34.401+0000 7f25eb200700 1 -- 172.21.15.50:0/1184754336 wait complete. 2024-05-10T21:05:35.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:34 smithi092 ceph-mon[34415]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T21:05:35.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:34 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1184754336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T21:05:35.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:34 smithi142 ceph-mon[34846]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T21:05:35.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:34 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1184754336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T21:05:35.144 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:34 smithi050 ceph-mon[29546]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T21:05:35.144 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:34 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1184754336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T21:05:35.217 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":628586267},{"type":"v1","addr":"172.21.15.50:6801","nonce":628586267}]},"active_addr":"172.21.15.50:6801/628586267","active_change":"2024-05-10T20:55:55.390513+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.50:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1669819871}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1753504381}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1781604349}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":914903729}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":425350897}]}]} 2024-05-10T21:05:35.223 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T21:05:35.223 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T21:05:35.223 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:05:35.747 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:36.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:36 smithi050 ceph-mon[29546]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:05:37.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:36 smithi092 ceph-mon[34415]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:05:37.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:36 smithi142 ceph-mon[34846]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 -- 172.21.15.50:0/2932901685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c075b60 msgr2=0x7f9c7c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 --2- 172.21.15.50:0/2932901685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c075b60 0x7f9c7c075f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009990 tx=0x7f9c70009ca0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 -- 172.21.15.50:0/2932901685 shutdown_connections 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 --2- 172.21.15.50:0/2932901685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c115370 0x7f9c7c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 --2- 172.21.15.50:0/2932901685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c075b60 0x7f9c7c075f70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 --2- 172.21.15.50:0/2932901685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 -- 172.21.15.50:0/2932901685 >> 172.21.15.50:0/2932901685 conn(0x7f9c7c0fffe0 msgr2=0x7f9c7c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 -- 172.21.15.50:0/2932901685 shutdown_connections 2024-05-10T21:05:37.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.811+0000 7f9c83400700 1 -- 172.21.15.50:0/2932901685 wait complete. 2024-05-10T21:05:37.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.813+0000 7f9c83400700 1 Processor -- start 2024-05-10T21:05:37.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.813+0000 7f9c83400700 1 -- start start 2024-05-10T21:05:37.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c115370 0x7f9c7c1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c11aa70 con 0x7f9c7c0764b0 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c11a8f0 con 0x7f9c7c115370 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c83400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c11abf0 con 0x7f9c7c075b60 2024-05-10T21:05:37.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c80a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c1a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52344/0 (socket says 172.21.15.50:52344) 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c80a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36562/0 (socket says 172.21.15.50:36562) 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c7be00700 1 -- 172.21.15.50:0/2945367448 learned_addr learned my addr 172.21.15.50:0/2945367448 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.814+0000 7f9c81400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c115370 0x7f9c7c1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 -- 172.21.15.50:0/2945367448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c115370 msgr2=0x7f9c7c1a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c115370 0x7f9c7c1a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 -- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 msgr2=0x7f9c7c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 -- 172.21.15.50:0/2945367448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c1a62b0 con 0x7f9c7c075b60 2024-05-10T21:05:37.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c80a00700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9c700059d0 tx=0x7f9c700047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70027070 con 0x7f9c7c075b60 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c1a6540 con 0x7f9c7c075b60 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c70022470 con 0x7f9c7c075b60 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.815+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70017650 con 0x7f9c7c075b60 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.816+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c1a6a40 con 0x7f9c7c075b60 2024-05-10T21:05:37.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.817+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c0512c0 con 0x7f9c7c075b60 2024-05-10T21:05:37.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.820+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9c7002a070 con 0x7f9c7c075b60 2024-05-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.821+0000 7f9c79600700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 0x7f9c640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.821+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9c700e39f0 con 0x7f9c7c075b60 2024-05-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.822+0000 7f9c7be00700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 0x7f9c640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.822+0000 7f9c7be00700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 0x7f9c640b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c103410 tx=0x7f9c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:37.823+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c70031070 con 0x7f9c7c075b60 2024-05-10T21:05:38.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.067+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c7c02e5f0 con 0x7f9c7c075b60 2024-05-10T21:05:38.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.069+0000 7f9c79600700 1 -- 172.21.15.50:0/2945367448 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7f9c7c1a6a40 con 0x7f9c7c075b60 2024-05-10T21:05:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:38.071 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 msgr2=0x7f9c640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 0x7f9c640b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c103410 tx=0x7f9c68009260 comp rx=0 tx=0).stop 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 msgr2=0x7f9c7c1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9c700059d0 tx=0x7f9c700047b0 comp rx=0 tx=0).stop 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 shutdown_connections 2024-05-10T21:05:38.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9c640b2990 0x7f9c640b4ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c115370 0x7f9c7c1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c7c075b60 0x7f9c7c1a13e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 --2- 172.21.15.50:0/2945367448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c7c0764b0 0x7f9c7c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.073+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 >> 172.21.15.50:0/2945367448 conn(0x7f9c7c0fffe0 msgr2=0x7f9c7c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.074+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 shutdown_connections 2024-05-10T21:05:38.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:38.074+0000 7f9c83400700 1 -- 172.21.15.50:0/2945367448 wait complete. 2024-05-10T21:05:38.923 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T21:05:38.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:05:39.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:38 smithi142 ceph-mon[34846]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:05:39.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:38 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2945367448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:39.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:38 smithi050 ceph-mon[29546]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:05:39.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:38 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2945367448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:38 smithi092 ceph-mon[34415]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:05:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:38 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2945367448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:41.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:40 smithi142 ceph-mon[34846]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:41.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:40 smithi050 ceph-mon[29546]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:41.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:40 smithi092 ceph-mon[34415]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 -- 172.21.15.50:0/2171805592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 msgr2=0x7fe6d8110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 --2- 172.21.15.50:0/2171805592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d8110dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009f90 tx=0x7fe6d000b330 comp rx=0 tx=0).stop 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 -- 172.21.15.50:0/2171805592 shutdown_connections 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 --2- 172.21.15.50:0/2171805592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 --2- 172.21.15.50:0/2171805592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d8110dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 --2- 172.21.15.50:0/2171805592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 -- 172.21.15.50:0/2171805592 >> 172.21.15.50:0/2171805592 conn(0x7fe6d80fef30 msgr2=0x7fe6d8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:41.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.544+0000 7fe6e0200700 1 -- 172.21.15.50:0/2171805592 shutdown_connections 2024-05-10T21:05:41.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.545+0000 7fe6e0200700 1 -- 172.21.15.50:0/2171805592 wait complete. 2024-05-10T21:05:41.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 Processor -- start 2024-05-10T21:05:41.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 -- start start 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8114450 con 0x7fe6d810e8f0 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81142d0 con 0x7fe6d8105480 2024-05-10T21:05:41.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.546+0000 7fe6e0200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81145d0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dd800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6de200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dd800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:51852/0 (socket says 172.21.15.50:51852) 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dd800700 1 -- 172.21.15.50:0/3863103241 learned_addr learned my addr 172.21.15.50:0/3863103241 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:41.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 -- 172.21.15.50:0/3863103241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 msgr2=0x7fe6d819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 -- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 msgr2=0x7fe6d81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dce00700 1 -- 172.21.15.50:0/3863103241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a1ef0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.547+0000 7fe6dd800700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6dce00700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d819d5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400e9c0 tx=0x7fe6c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400c9c0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a21e0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c400cb60 con 0x7fe6d8105dd0 2024-05-10T21:05:41.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4018420 con 0x7fe6d8105dd0 2024-05-10T21:05:41.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.548+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a9b00 con 0x7fe6d8105dd0 2024-05-10T21:05:41.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.549+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8005330 con 0x7fe6d8105dd0 2024-05-10T21:05:41.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.552+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe6c4010430 con 0x7fe6d8105dd0 2024-05-10T21:05:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.554+0000 7fe6d6000700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 0x7fe6c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.554+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fe6c40e2820 con 0x7fe6d8105dd0 2024-05-10T21:05:41.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.554+0000 7fe6dd800700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 0x7fe6c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:41.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.555+0000 7fe6dd800700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 0x7fe6c00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc011550 tx=0x7fe6cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:41.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.556+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6c40aa3b0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.797+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b80058f0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.800+0000 7fe6d6000700 1 -- 172.21.15.50:0/3863103241 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7fe6c40a9cb0 con 0x7fe6d8105dd0 2024-05-10T21:05:41.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:05:41.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 msgr2=0x7fe6c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 0x7fe6c00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc011550 tx=0x7fe6cc009260 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 msgr2=0x7fe6d819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d819d5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400e9c0 tx=0x7fe6c400ecd0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 shutdown_connections 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6c00b2910 0x7fe6c00b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105480 0x7fe6d819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6d810e8f0 0x7fe6d81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 --2- 172.21.15.50:0/3863103241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d8105dd0 0x7fe6d819d5c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 >> 172.21.15.50:0/3863103241 conn(0x7fe6d80fef30 msgr2=0x7fe6d8103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 shutdown_connections 2024-05-10T21:05:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:41.803+0000 7fe6e0200700 1 -- 172.21.15.50:0/3863103241 wait complete. 2024-05-10T21:05:42.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:41 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3863103241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:42.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:41 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3863103241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:42.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:41 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3863103241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:05:42.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T21:05:42.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T21:05:42.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T21:05:42.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T21:05:42.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T21:05:42.628 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T21:05:42.628 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T21:05:42.629 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T21:05:42.629 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T21:05:42.629 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T21:05:42.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T21:05:42.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T21:05:43.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:42 smithi050 ceph-mon[29546]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:43.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:42 smithi092 ceph-mon[34415]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:43.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:42 smithi142 ceph-mon[34846]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:43.628 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.114 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.126 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.132 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.141 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.141 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.142 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.142 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.142 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.149 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.152 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:44.168 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:05:45.171 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:44 smithi050 ceph-mon[29546]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:45.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:44 smithi092 ceph-mon[34415]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:45.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:44 smithi142 ceph-mon[34846]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:47.025 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:46 smithi050 ceph-mon[29546]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:47.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:46 smithi092 ceph-mon[34415]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:47.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:46 smithi142 ceph-mon[34846]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:48.980 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:48 smithi050 ceph-mon[29546]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:49.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:48 smithi092 ceph-mon[34415]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:49.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:48 smithi142 ceph-mon[34846]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:50.272 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:49 smithi092 ceph-mon[34415]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:50.372 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:49 smithi050 ceph-mon[29546]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:50.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:49 smithi142 ceph-mon[34846]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:52.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:52 smithi050 ceph-mon[29546]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:53.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:52 smithi092 ceph-mon[34415]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:53.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:52 smithi142 ceph-mon[34846]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:05:54.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 -- 172.21.15.50:0/3726878434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790108f20 msgr2=0x7ff790109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 --2- 172.21.15.50:0/3726878434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790108f20 0x7ff790109330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0099c0 tx=0x7ff77c009cd0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 -- 172.21.15.50:0/3726878434 shutdown_connections 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 --2- 172.21.15.50:0/3726878434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790109870 0x7ff7901111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 --2- 172.21.15.50:0/3726878434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7901045f0 0x7ff790104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 --2- 172.21.15.50:0/3726878434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790108f20 0x7ff790109330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 -- 172.21.15.50:0/3726878434 >> 172.21.15.50:0/3726878434 conn(0x7ff790100040 msgr2=0x7ff790102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 -- 172.21.15.50:0/3726878434 shutdown_connections 2024-05-10T21:05:54.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.537+0000 7ff796800700 1 -- 172.21.15.50:0/3726878434 wait complete. 2024-05-10T21:05:54.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.539+0000 7ff796800700 1 Processor -- start 2024-05-10T21:05:54.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.539+0000 7ff796800700 1 -- start start 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7901045f0 0x7ff79019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 0x7ff79019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790109870 0x7ff7901a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901144d0 con 0x7ff790109870 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff790114350 con 0x7ff790108f20 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff796800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff790114650 con 0x7ff7901045f0 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790109870 0x7ff7901a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 0x7ff79019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7901045f0 0x7ff79019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 0x7ff79019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39238/0 (socket says 172.21.15.50:39238) 2024-05-10T21:05:54.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790109870 0x7ff7901a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55470/0 (socket says 172.21.15.50:55470) 2024-05-10T21:05:54.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 -- 172.21.15.50:0/1797501818 learned_addr learned my addr 172.21.15.50:0/1797501818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 -- 172.21.15.50:0/1797501818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7901045f0 msgr2=0x7ff79019d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7901045f0 0x7ff79019d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 -- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790109870 msgr2=0x7ff7901a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff790109870 0x7ff7901a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.540+0000 7ff78ea00700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7901a1f50 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff78ea00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 0x7ff79019d620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00b580 tx=0x7ff77c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c009e60 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77c003c70 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7901a21e0 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004910 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.541+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7901a2660 con 0x7ff790108f20 2024-05-10T21:05:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.542+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff790076d10 con 0x7ff790108f20 2024-05-10T21:05:54.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.545+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff77c0037c0 con 0x7ff790108f20 2024-05-10T21:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff787e00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7740b2910 0x7ff7740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff77c0e3370 con 0x7ff790108f20 2024-05-10T21:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff78f400700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7740b2910 0x7ff7740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff787e00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] conn(0x7ff7740bc270 0x7ff7740be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff77c00b0b0 con 0x7ff7740bc270 2024-05-10T21:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.547+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff77c0e3720 con 0x7ff790108f20 2024-05-10T21:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.548+0000 7ff78fe00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] conn(0x7ff7740bc270 0x7ff7740be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.548+0000 7ff78f400700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7740b2910 0x7ff7740b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff780009c60 tx=0x7ff780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.548+0000 7ff78fe00700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] conn(0x7ff7740bc270 0x7ff7740be760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.550+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== osd.4 v2:172.21.15.92:6808/1024551638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff77c00b0b0 con 0x7ff7740bc270 2024-05-10T21:05:54.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.568+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 --> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff790105aa0 con 0x7ff7740bc270 2024-05-10T21:05:54.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.568+0000 7ff787e00700 1 -- 172.21.15.50:0/1797501818 <== osd.4 v2:172.21.15.92:6808/1024551638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff790105aa0 con 0x7ff7740bc270 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] conn(0x7ff7740bc270 msgr2=0x7ff7740be760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] conn(0x7ff7740bc270 0x7ff7740be760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7740b2910 msgr2=0x7ff7740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7740b2910 0x7ff7740b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff780009c60 tx=0x7ff780009260 comp rx=0 tx=0).stop 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 msgr2=0x7ff79019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 --2- 172.21.15.50:0/1797501818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff790108f20 0x7ff79019d620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00b580 tx=0x7ff77c004270 comp rx=0 tx=0).stop 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff78fe00700 1 -- 172.21.15.50:0/1797501818 reap_dead start 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 shutdown_connections 2024-05-10T21:05:54.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.570+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 >> 172.21.15.50:0/1797501818 conn(0x7ff790100040 msgr2=0x7ff79010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:54.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.571+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 shutdown_connections 2024-05-10T21:05:54.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.571+0000 7ff796800700 1 -- 172.21.15.50:0/1797501818 wait complete. 2024-05-10T21:05:54.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:54 smithi050 ceph-mon[29546]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:54.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.821+0000 7f8082600700 1 -- 172.21.15.50:0/915786349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c1109b0 msgr2=0x7f807c112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.821+0000 7f8082600700 1 --2- 172.21.15.50:0/915786349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c1109b0 0x7f807c112e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8074009f90 tx=0x7f807400b330 comp rx=0 tx=0).stop 2024-05-10T21:05:54.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 -- 172.21.15.50:0/915786349 shutdown_connections 2024-05-10T21:05:54.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 --2- 172.21.15.50:0/915786349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c075d30 0x7f807c076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 --2- 172.21.15.50:0/915786349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c1109b0 0x7f807c112e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 --2- 172.21.15.50:0/915786349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f807c076680 0x7f807c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 -- 172.21.15.50:0/915786349 >> 172.21.15.50:0/915786349 conn(0x7f807c079d10 msgr2=0x7f807c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:54.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 -- 172.21.15.50:0/915786349 shutdown_connections 2024-05-10T21:05:54.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.822+0000 7f8082600700 1 -- 172.21.15.50:0/915786349 wait complete. 2024-05-10T21:05:54.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.823+0000 7f8082600700 1 Processor -- start 2024-05-10T21:05:54.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.823+0000 7f8082600700 1 -- start start 2024-05-10T21:05:54.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 0x7f807c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c076680 0x7f807c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f807c1109b0 0x7f807c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f807c116490 con 0x7f807c075d30 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f807c116310 con 0x7f807c076680 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8082600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f807c116610 con 0x7f807c1109b0 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8081200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 0x7f807c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8081200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 0x7f807c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55490/0 (socket says 172.21.15.50:55490) 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f8081200700 1 -- 172.21.15.50:0/3740712345 learned_addr learned my addr 172.21.15.50:0/3740712345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.824+0000 7f807be00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c076680 0x7f807c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081c00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f807c1109b0 0x7f807c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 -- 172.21.15.50:0/3740712345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f807c1109b0 msgr2=0x7f807c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f807c1109b0 0x7f807c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 -- 172.21.15.50:0/3740712345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c076680 msgr2=0x7f807c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c076680 0x7f807c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c1a1cc0 con 0x7f807c075d30 2024-05-10T21:05:54.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f807be00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f807c076680 0x7f807c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:54.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8081200700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 0x7f807c19ce40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9c0 tx=0x7f807000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070004a30 con 0x7f807c075d30 2024-05-10T21:05:54.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807000cc70 con 0x7f807c075d30 2024-05-10T21:05:54.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070005490 con 0x7f807c075d30 2024-05-10T21:05:54.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.825+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1a1fb0 con 0x7f807c075d30 2024-05-10T21:05:54.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.826+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1a98d0 con 0x7f807c075d30 2024-05-10T21:05:54.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.827+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f807c063f50 con 0x7f807c075d30 2024-05-10T21:05:54.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.829+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8070004bd0 con 0x7f807c075d30 2024-05-10T21:05:54.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f8079600700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f80640b2910 0x7f80640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f80700e28c0 con 0x7f807c075d30 2024-05-10T21:05:54.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f807be00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f80640b2910 0x7f80640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f8079600700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] conn(0x7f80640bc3a0 0x7f80640be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:54.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f807000be50 con 0x7f80640bc3a0 2024-05-10T21:05:54.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.831+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f80700e2c70 con 0x7f807c075d30 2024-05-10T21:05:54.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.832+0000 7f8081c00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] conn(0x7f80640bc3a0 0x7f80640be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:54.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.832+0000 7f807be00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f80640b2910 0x7f80640b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f807c1128b0 tx=0x7f8068009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.832+0000 7f8081c00700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] conn(0x7f80640bc3a0 0x7f80640be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:54.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.833+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== osd.6 v2:172.21.15.50:6818/2895143943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f807000be50 con 0x7f80640bc3a0 2024-05-10T21:05:54.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.852+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 --> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f807c0643d0 con 0x7f80640bc3a0 2024-05-10T21:05:54.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.852+0000 7f8079600700 1 -- 172.21.15.50:0/3740712345 <== osd.6 v2:172.21.15.50:6818/2895143943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f807c0643d0 con 0x7f80640bc3a0 2024-05-10T21:05:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] conn(0x7f80640bc3a0 msgr2=0x7f80640be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] conn(0x7f80640bc3a0 0x7f80640be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f80640b2910 msgr2=0x7f80640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f80640b2910 0x7f80640b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f807c1128b0 tx=0x7f8068009260 comp rx=0 tx=0).stop 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 msgr2=0x7f807c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 --2- 172.21.15.50:0/3740712345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f807c075d30 0x7f807c19ce40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9c0 tx=0x7f807000ecd0 comp rx=0 tx=0).stop 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8081c00700 1 -- 172.21.15.50:0/3740712345 reap_dead start 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 shutdown_connections 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 >> 172.21.15.50:0/3740712345 conn(0x7f807c079d10 msgr2=0x7f807c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 shutdown_connections 2024-05-10T21:05:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:54.853+0000 7f8082600700 1 -- 172.21.15.50:0/3740712345 wait complete. 2024-05-10T21:05:55.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:54 smithi092 ceph-mon[34415]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:55.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:54 smithi142 ceph-mon[34846]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:05:56.921 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:56 smithi050 ceph-mon[29546]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:05:57.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:56 smithi092 ceph-mon[34415]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:05:57.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:56 smithi142 ceph-mon[34846]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:05:58.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 -- 172.21.15.50:0/2466021493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106ed0 msgr2=0x7fbff8113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 --2- 172.21.15.50:0/2466021493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106ed0 0x7fbff8113360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b0a0 tx=0x7fbff000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 -- 172.21.15.50:0/2466021493 shutdown_connections 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 --2- 172.21.15.50:0/2466021493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 0x7fbff810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 --2- 172.21.15.50:0/2466021493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106ed0 0x7fbff8113360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 --2- 172.21.15.50:0/2466021493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106580 0x7fbff8106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 -- 172.21.15.50:0/2466021493 >> 172.21.15.50:0/2466021493 conn(0x7fbff8102080 msgr2=0x7fbff81044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 -- 172.21.15.50:0/2466021493 shutdown_connections 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.422+0000 7fbfff800700 1 -- 172.21.15.50:0/2466021493 wait complete. 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.423+0000 7fbfff800700 1 Processor -- start 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.423+0000 7fbfff800700 1 -- start start 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.423+0000 7fbfff800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106580 0x7fbff819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:58.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbfff800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 0x7fbff819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbfff800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 0x7fbff81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbfff800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81166f0 con 0x7fbff8106580 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbfff800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8116570 con 0x7fbff810c5c0 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbfff800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8116870 con 0x7fbff8106ed0 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffce00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106580 0x7fbff819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 0x7fbff819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 0x7fbff819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32962/0 (socket says 172.21.15.50:32962) 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffce00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106580 0x7fbff819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55512/0 (socket says 172.21.15.50:55512) 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbff7e00700 1 -- 172.21.15.50:0/433982085 learned_addr learned my addr 172.21.15.50:0/433982085 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 0x7fbff81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 -- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 msgr2=0x7fbff819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 0x7fbff819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 -- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106580 msgr2=0x7fbff819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbff8106580 0x7fbff819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbffd800700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff81a1e90 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.424+0000 7fbff7e00700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbff8106ed0 0x7fbff819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbffd800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 0x7fbff81a17f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b070 tx=0x7fbff0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0021070 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff81a2120 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff0007e30 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00045d0 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.425+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81a9aa0 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.426+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbff8063f10 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.429+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbff0003c00 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbff5600700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbfe00b2910 0x7fbfe00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fbff000f020 con 0x7fbff810c5c0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbffce00700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbfe00b2910 0x7fbfe00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:58.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbff5600700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] conn(0x7fbfe00bc3a0 0x7fbfe00be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbff0029e70 con 0x7fbfe00bc3a0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.431+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fbff00e3c60 con 0x7fbff810c5c0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.432+0000 7fbff7e00700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] conn(0x7fbfe00bc3a0 0x7fbfe00be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.432+0000 7fbffce00700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbfe00b2910 0x7fbfe00b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbff81079f0 tx=0x7fbfec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.432+0000 7fbff7e00700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] conn(0x7fbfe00bc3a0 0x7fbfe00be890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.433+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== osd.2 v2:172.21.15.142:6800/1105819435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbff0029e70 con 0x7fbfe00bc3a0 2024-05-10T21:05:58.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.452+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 --> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbff8064310 con 0x7fbfe00bc3a0 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.452+0000 7fbff5600700 1 -- 172.21.15.50:0/433982085 <== osd.2 v2:172.21.15.142:6800/1105819435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbff8064310 con 0x7fbfe00bc3a0 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] conn(0x7fbfe00bc3a0 msgr2=0x7fbfe00be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] conn(0x7fbfe00bc3a0 0x7fbfe00be890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbfe00b2910 msgr2=0x7fbfe00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbfe00b2910 0x7fbfe00b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbff81079f0 tx=0x7fbfec009260 comp rx=0 tx=0).stop 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 msgr2=0x7fbff81a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 --2- 172.21.15.50:0/433982085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff810c5c0 0x7fbff81a17f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbff000b070 tx=0x7fbff0004d00 comp rx=0 tx=0).stop 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbffd800700 1 -- 172.21.15.50:0/433982085 reap_dead start 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 shutdown_connections 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 >> 172.21.15.50:0/433982085 conn(0x7fbff8102080 msgr2=0x7fbff8104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 shutdown_connections 2024-05-10T21:05:58.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:57.453+0000 7fbfff800700 1 -- 172.21.15.50:0/433982085 wait complete. 2024-05-10T21:05:58.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:05:58 smithi050 ceph-mon[29546]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:59.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:05:58 smithi092 ceph-mon[34415]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:05:58 smithi142 ceph-mon[34846]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 -- 172.21.15.50:0/2443690331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10a7c0 msgr2=0x7fa31c10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 --2- 172.21.15.50:0/2443690331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10a7c0 0x7fa31c10abd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa3080098f0 tx=0x7fa308009c00 comp rx=0 tx=0).stop 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 -- 172.21.15.50:0/2443690331 shutdown_connections 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 --2- 172.21.15.50:0/2443690331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10b110 0x7fa31c117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 --2- 172.21.15.50:0/2443690331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 --2- 172.21.15.50:0/2443690331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10a7c0 0x7fa31c10abd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 -- 172.21.15.50:0/2443690331 >> 172.21.15.50:0/2443690331 conn(0x7fa31c076b00 msgr2=0x7fa31c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:59.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 -- 172.21.15.50:0/2443690331 shutdown_connections 2024-05-10T21:05:59.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.795+0000 7fa322a00700 1 -- 172.21.15.50:0/2443690331 wait complete. 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.796+0000 7fa322a00700 1 Processor -- start 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 -- start start 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10a7c0 0x7fa31c1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10b110 0x7fa31c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c11a900 con 0x7fa31c10f6c0 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c11a780 con 0x7fa31c10b110 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.797+0000 7fa322a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c11aa80 con 0x7fa31c10a7c0 2024-05-10T21:05:59.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa322000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:59.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa321600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10a7c0 0x7fa31c1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:59.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10b110 0x7fa31c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:59.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa322000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c1a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55526/0 (socket says 172.21.15.50:55526) 2024-05-10T21:05:59.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa322000700 1 -- 172.21.15.50:0/3363093749 learned_addr learned my addr 172.21.15.50:0/3363093749 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:05:59.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa321600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10a7c0 0x7fa31c1a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32978/0 (socket says 172.21.15.50:32978) 2024-05-10T21:05:59.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 -- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10a7c0 msgr2=0x7fa31c1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31c10a7c0 0x7fa31c1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 -- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 msgr2=0x7fa31c1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1a6180 con 0x7fa31c10b110 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa322000700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c10f6c0 0x7fa31c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.798+0000 7fa320c00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10b110 0x7fa31c1a1850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c010 tx=0x7fa30800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.799+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080051c0 con 0x7fa31c10b110 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.799+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30800b560 con 0x7fa31c10b110 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.799+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308003970 con 0x7fa31c10b110 2024-05-10T21:05:59.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.799+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1a6410 con 0x7fa31c10b110 2024-05-10T21:05:59.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.799+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a68f0 con 0x7fa31c10b110 2024-05-10T21:05:59.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.800+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa31c063f50 con 0x7fa31c10b110 2024-05-10T21:05:59.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.802+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa30800b930 con 0x7fa31c10b110 2024-05-10T21:05:59.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa31a000700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3040b2910 0x7fa3040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:59.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa3080e35e0 con 0x7fa31c10b110 2024-05-10T21:05:59.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa321600700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3040b2910 0x7fa3040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:59.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa31a000700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] conn(0x7fa3040bc3a0 0x7fa3040be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:05:59.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa30801fe80 con 0x7fa3040bc3a0 2024-05-10T21:05:59.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.804+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa3080e3990 con 0x7fa31c10b110 2024-05-10T21:05:59.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.805+0000 7fa322000700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] conn(0x7fa3040bc3a0 0x7fa3040be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:05:59.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.805+0000 7fa321600700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3040b2910 0x7fa3040b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa310005fd0 tx=0x7fa310009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:59.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.805+0000 7fa322000700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] conn(0x7fa3040bc3a0 0x7fa3040be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:05:59.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.806+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== osd.8 v2:172.21.15.142:6816/2574916592 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30801fe80 con 0x7fa3040bc3a0 2024-05-10T21:05:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.825+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 --> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa31c064380 con 0x7fa3040bc3a0 2024-05-10T21:05:59.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa31a000700 1 -- 172.21.15.50:0/3363093749 <== osd.8 v2:172.21.15.142:6816/2574916592 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa31c064380 con 0x7fa3040bc3a0 2024-05-10T21:05:59.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] conn(0x7fa3040bc3a0 msgr2=0x7fa3040be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] conn(0x7fa3040bc3a0 0x7fa3040be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:05:59.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3040b2910 msgr2=0x7fa3040b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3040b2910 0x7fa3040b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa310005fd0 tx=0x7fa310009310 comp rx=0 tx=0).stop 2024-05-10T21:05:59.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10b110 msgr2=0x7fa31c1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:05:59.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.826+0000 7fa322a00700 1 --2- 172.21.15.50:0/3363093749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31c10b110 0x7fa31c1a1850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c010 tx=0x7fa30800fa00 comp rx=0 tx=0).stop 2024-05-10T21:05:59.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.827+0000 7fa322000700 1 -- 172.21.15.50:0/3363093749 reap_dead start 2024-05-10T21:05:59.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.827+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 shutdown_connections 2024-05-10T21:05:59.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.827+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 >> 172.21.15.50:0/3363093749 conn(0x7fa31c076b00 msgr2=0x7fa31c1088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:05:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.827+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 shutdown_connections 2024-05-10T21:05:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:05:59.827+0000 7fa322a00700 1 -- 172.21.15.50:0/3363093749 wait complete. 2024-05-10T21:06:00.846 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:00 smithi050 ceph-mon[29546]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:00 smithi092 ceph-mon[34415]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:01.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:00 smithi142 ceph-mon[34846]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:01.290 INFO:teuthology.orchestra.run.smithi050.stdout:133143986254 2024-05-10T21:06:01.291 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 -- 172.21.15.50:0/2312308399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148076e40 msgr2=0x7fa148077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 --2- 172.21.15.50:0/2312308399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148076e40 0x7fa148077250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 -- 172.21.15.50:0/2312308399 shutdown_connections 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 --2- 172.21.15.50:0/2312308399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148110b90 0x7fa148112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 --2- 172.21.15.50:0/2312308399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148076e40 0x7fa148077250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 --2- 172.21.15.50:0/2312308399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa148077790 0x7fa148110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 -- 172.21.15.50:0/2312308399 >> 172.21.15.50:0/2312308399 conn(0x7fa1480ffef0 msgr2=0x7fa148102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:01.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.813+0000 7fa14ce00700 1 -- 172.21.15.50:0/2312308399 shutdown_connections 2024-05-10T21:06:01.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.814+0000 7fa14ce00700 1 -- 172.21.15.50:0/2312308399 wait complete. 2024-05-10T21:06:01.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 Processor -- start 2024-05-10T21:06:01.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 -- start start 2024-05-10T21:06:01.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 0x7fa14819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:01.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa148077790 0x7fa14819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148110b90 0x7fa1481a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa148116680 con 0x7fa148110b90 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa148116500 con 0x7fa148076e40 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.815+0000 7fa14ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa148116800 con 0x7fa148077790 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 0x7fa14819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 0x7fa14819d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39314/0 (socket says 172.21.15.50:39314) 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 -- 172.21.15.50:0/563074385 learned_addr learned my addr 172.21.15.50:0/563074385 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa146800700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148110b90 0x7fa1481a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145400700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa148077790 0x7fa14819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 -- 172.21.15.50:0/563074385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa148077790 msgr2=0x7fa14819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa148077790 0x7fa14819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 -- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148110b90 msgr2=0x7fa1481a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa148110b90 0x7fa1481a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a1f00 con 0x7fa148076e40 2024-05-10T21:06:01.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.816+0000 7fa145e00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 0x7fa14819d030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1380057f0 tx=0x7fa13800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.817+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138026070 con 0x7fa148076e40 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.817+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa138021410 con 0x7fa148076e40 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.817+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a2190 con 0x7fa148076e40 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.817+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138017650 con 0x7fa148076e40 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.817+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a9b10 con 0x7fa148076e40 2024-05-10T21:06:01.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.818+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa148063f50 con 0x7fa148076e40 2024-05-10T21:06:01.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.820+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa138029070 con 0x7fa148076e40 2024-05-10T21:06:01.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.822+0000 7fa13ea00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa12c0b2990 0x7fa12c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:01.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.822+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa1380e35b0 con 0x7fa148076e40 2024-05-10T21:06:01.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.822+0000 7fa145400700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa12c0b2990 0x7fa12c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:01.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.823+0000 7fa13ea00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] conn(0x7fa12c0bc420 0x7fa12c0be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:01.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.823+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa138001d30 con 0x7fa12c0bc420 2024-05-10T21:06:01.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.823+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa1380e3960 con 0x7fa148076e40 2024-05-10T21:06:01.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.823+0000 7fa145400700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa12c0b2990 0x7fa12c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1481033f0 tx=0x7fa134009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:01.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.823+0000 7fa146800700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] conn(0x7fa12c0bc420 0x7fa12c0be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:01.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.824+0000 7fa146800700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] conn(0x7fa12c0bc420 0x7fa12c0be910 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:01.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.825+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== osd.10 v2:172.21.15.92:6824/955638920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa138001d30 con 0x7fa12c0bc420 2024-05-10T21:06:01.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.843+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 --> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa148064350 con 0x7fa12c0bc420 2024-05-10T21:06:01.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa13ea00700 1 -- 172.21.15.50:0/563074385 <== osd.10 v2:172.21.15.92:6824/955638920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa148064350 con 0x7fa12c0bc420 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] conn(0x7fa12c0bc420 msgr2=0x7fa12c0be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] conn(0x7fa12c0bc420 0x7fa12c0be910 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa12c0b2990 msgr2=0x7fa12c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa12c0b2990 0x7fa12c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1481033f0 tx=0x7fa134009260 comp rx=0 tx=0).stop 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 msgr2=0x7fa14819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa14ce00700 1 --2- 172.21.15.50:0/563074385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa148076e40 0x7fa14819d030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1380057f0 tx=0x7fa13800bfd0 comp rx=0 tx=0).stop 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.844+0000 7fa146800700 1 -- 172.21.15.50:0/563074385 reap_dead start 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.845+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 shutdown_connections 2024-05-10T21:06:01.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.845+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 >> 172.21.15.50:0/563074385 conn(0x7fa1480ffef0 msgr2=0x7fa148100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:01.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.845+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 shutdown_connections 2024-05-10T21:06:01.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.845+0000 7fa14ce00700 1 -- 172.21.15.50:0/563074385 wait complete. 2024-05-10T21:06:02.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 -- 172.21.15.50:0/1842369948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106ec0 msgr2=0x7f580c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 --2- 172.21.15.50:0/1842369948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106ec0 0x7f580c113350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580400b0a0 tx=0x7f580400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 -- 172.21.15.50:0/1842369948 shutdown_connections 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 --2- 172.21.15.50:0/1842369948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 0x7f580c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 --2- 172.21.15.50:0/1842369948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106ec0 0x7f580c113350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 --2- 172.21.15.50:0/1842369948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f580c106570 0x7f580c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 -- 172.21.15.50:0/1842369948 >> 172.21.15.50:0/1842369948 conn(0x7f580c102050 msgr2=0x7f580c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 -- 172.21.15.50:0/1842369948 shutdown_connections 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.388+0000 7f5811a00700 1 -- 172.21.15.50:0/1842369948 wait complete. 2024-05-10T21:06:02.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 Processor -- start 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 -- start start 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 0x7f580c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f580c106ec0 0x7f580c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 0x7f580c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f580c116730 con 0x7f580c106ec0 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1165b0 con 0x7f580c10c5b0 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.390+0000 7f5811a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1168b0 con 0x7f580c106570 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 0x7f580c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 0x7f580c19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32986/0 (socket says 172.21.15.50:32986) 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 -- 172.21.15.50:0/4067700494 learned_addr learned my addr 172.21.15.50:0/4067700494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580a000700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f580c106ec0 0x7f580c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:02.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580b400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 0x7f580c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 -- 172.21.15.50:0/4067700494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 msgr2=0x7f580c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 0x7f580c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 -- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f580c106ec0 msgr2=0x7f580c19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f580c106ec0 0x7f580c19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580aa00700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1a1ed0 con 0x7f580c106570 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.391+0000 7f580b400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f580c10c5b0 0x7f580c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f580aa00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 0x7f580c19d000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e970 tx=0x7f57fc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:02.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00cbd0 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57fc00cd70 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c1a21c0 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc005680 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.392+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c1a9ae0 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.393+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f580c063f10 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.396+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f57fc004750 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.397+0000 7f5803400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f57f00b2990 0x7f57f00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f57fc0e29b0 con 0x7f580c106570 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f580a000700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f57f00b2990 0x7f57f00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f5803400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] conn(0x7f57f00bc420 0x7f57f00be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:02.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57fc01aec0 con 0x7f57f00bc420 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0e2d60 con 0x7f580c106570 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.398+0000 7f580b400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] conn(0x7f57f00bc420 0x7f57f00be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.399+0000 7f580a000700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f57f00b2990 0x7f57f00b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f580c1054b0 tx=0x7f57f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.399+0000 7f580b400700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] conn(0x7f57f00bc420 0x7f57f00be910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.400+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== osd.11 v2:172.21.15.142:6824/256997334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57fc01aec0 con 0x7f57f00bc420 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.419+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 --> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f580c064310 con 0x7f57f00bc420 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.419+0000 7f5803400700 1 -- 172.21.15.50:0/4067700494 <== osd.11 v2:172.21.15.142:6824/256997334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f580c064310 con 0x7f57f00bc420 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] conn(0x7f57f00bc420 msgr2=0x7f57f00be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] conn(0x7f57f00bc420 0x7f57f00be910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f57f00b2990 msgr2=0x7f57f00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f57f00b2990 0x7f57f00b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f580c1054b0 tx=0x7f57f8009260 comp rx=0 tx=0).stop 2024-05-10T21:06:02.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 msgr2=0x7f580c19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 --2- 172.21.15.50:0/4067700494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580c106570 0x7f580c19d000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e970 tx=0x7f57fc00ec80 comp rx=0 tx=0).stop 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f580b400700 1 -- 172.21.15.50:0/4067700494 reap_dead start 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 shutdown_connections 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.420+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 >> 172.21.15.50:0/4067700494 conn(0x7f580c102050 msgr2=0x7f580c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.421+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 shutdown_connections 2024-05-10T21:06:02.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:01.421+0000 7f5811a00700 1 -- 172.21.15.50:0/4067700494 wait complete. 2024-05-10T21:06:02.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:02 smithi050 ceph-mon[29546]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:02.953 INFO:teuthology.orchestra.run.smithi050.stdout:77309411417 2024-05-10T21:06:02.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T21:06:03.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:02 smithi092 ceph-mon[34415]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:03.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:02 smithi142 ceph-mon[34846]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 -- 172.21.15.50:0/645026021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70108f20 msgr2=0x7f5f70109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 --2- 172.21.15.50:0/645026021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70108f20 0x7f5f70109330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0099c0 tx=0x7f5f5c009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 -- 172.21.15.50:0/645026021 shutdown_connections 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 --2- 172.21.15.50:0/645026021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f70109870 0x7f5f701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 --2- 172.21.15.50:0/645026021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 0x7f5f70104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 --2- 172.21.15.50:0/645026021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70108f20 0x7f5f70109330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 -- 172.21.15.50:0/645026021 >> 172.21.15.50:0/645026021 conn(0x7f5f70100040 msgr2=0x7f5f70102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 -- 172.21.15.50:0/645026021 shutdown_connections 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.136+0000 7f5f77200700 1 -- 172.21.15.50:0/645026021 wait complete. 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.137+0000 7f5f77200700 1 Processor -- start 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.137+0000 7f5f77200700 1 -- start start 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 0x7f5f7019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f70108f20 0x7f5f7019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70109870 0x7f5f701a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701144d0 con 0x7f5f70109870 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70114350 con 0x7f5f70108f20 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f77200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70114650 con 0x7f5f701045f0 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f75200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70109870 0x7f5f701a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f6fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 0x7f5f7019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f75200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70109870 0x7f5f701a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55592/0 (socket says 172.21.15.50:55592) 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.138+0000 7f5f75200700 1 -- 172.21.15.50:0/4006843823 learned_addr learned my addr 172.21.15.50:0/4006843823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6f400700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f70108f20 0x7f5f7019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 -- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f70108f20 msgr2=0x7f5f7019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f70108f20 0x7f5f7019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 -- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70109870 msgr2=0x7f5f701a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f70109870 0x7f5f701a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f701a1f50 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.139+0000 7f5f6fe00700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 0x7f5f7019d080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9f0 tx=0x7f5f6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.140+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000c9c0 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.140+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f701a2240 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.140+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6000cb60 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.140+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60018680 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.140+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f701a9b60 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.141+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f70076d10 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.144+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5f60010430 con 0x7f5f701045f0 2024-05-10T21:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6cc00700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5f540b2990 0x7f5f540b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5f600e2ab0 con 0x7f5f701045f0 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6f400700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5f540b2990 0x7f5f540b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6cc00700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] conn(0x7f5f540bc420 0x7f5f540be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f540befe0 con 0x7f5f540bc420 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.147+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5f600e2e60 con 0x7f5f701045f0 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.148+0000 7f5f6f400700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5f540b2990 0x7f5f540b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000c00 tx=0x7f5f5c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.148+0000 7f5f75200700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] conn(0x7f5f540bc420 0x7f5f540be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.148+0000 7f5f75200700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] conn(0x7f5f540bc420 0x7f5f540be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.149+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== osd.1 v2:172.21.15.92:6800/1268741901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f540befe0 con 0x7f5f540bc420 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.169+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 --> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f70105aa0 con 0x7f5f540bc420 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.170+0000 7f5f6cc00700 1 -- 172.21.15.50:0/4006843823 <== osd.1 v2:172.21.15.92:6800/1268741901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5f70105aa0 con 0x7f5f540bc420 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.170+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] conn(0x7f5f540bc420 msgr2=0x7f5f540be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.170+0000 7f5f77200700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] conn(0x7f5f540bc420 0x7f5f540be910 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5f540b2990 msgr2=0x7f5f540b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5f540b2990 0x7f5f540b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000c00 tx=0x7f5f5c01a040 comp rx=0 tx=0).stop 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 msgr2=0x7f5f7019d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 --2- 172.21.15.50:0/4006843823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f701045f0 0x7f5f7019d080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9f0 tx=0x7f5f6000ed00 comp rx=0 tx=0).stop 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f75200700 1 -- 172.21.15.50:0/4006843823 reap_dead start 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 shutdown_connections 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 >> 172.21.15.50:0/4006843823 conn(0x7f5f70100040 msgr2=0x7f5f7010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:03.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 shutdown_connections 2024-05-10T21:06:03.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.171+0000 7f5f77200700 1 -- 172.21.15.50:0/4006843823 wait complete. 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.984+0000 7f155b400700 1 -- 172.21.15.50:0/661434168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554075b00 msgr2=0x7f1554075f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.984+0000 7f155b400700 1 --2- 172.21.15.50:0/661434168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554075b00 0x7f1554075f10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1548009990 tx=0x7f1548009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 -- 172.21.15.50:0/661434168 shutdown_connections 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 --2- 172.21.15.50:0/661434168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554110c00 0x7f1554113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 --2- 172.21.15.50:0/661434168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554075b00 0x7f1554075f10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 --2- 172.21.15.50:0/661434168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554076450 0x7f15541106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 -- 172.21.15.50:0/661434168 >> 172.21.15.50:0/661434168 conn(0x7f1554100040 msgr2=0x7f1554102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 -- 172.21.15.50:0/661434168 shutdown_connections 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.985+0000 7f155b400700 1 -- 172.21.15.50:0/661434168 wait complete. 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 Processor -- start 2024-05-10T21:06:04.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 -- start start 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554075b00 0x7f155419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 0x7f155419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554110c00 0x7f15541a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15541166e0 con 0x7f1554110c00 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1554116560 con 0x7f1554075b00 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.986+0000 7f155b400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1554116860 con 0x7f1554076450 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 0x7f155419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1559400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554110c00 0x7f15541a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 0x7f155419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60164/0 (socket says 172.21.15.50:60164) 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 -- 172.21.15.50:0/190721897 learned_addr learned my addr 172.21.15.50:0/190721897 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1558a00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554075b00 0x7f155419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 -- 172.21.15.50:0/190721897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554075b00 msgr2=0x7f155419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554075b00 0x7f155419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 -- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554110c00 msgr2=0x7f15541a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1554110c00 0x7f15541a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1553e00700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15541a1f00 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.987+0000 7f1558a00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1554075b00 0x7f155419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f1553e00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 0x7f155419d5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f154000e970 tx=0x7f154000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540004a30 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15541a21f0 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154000cc90 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400054c0 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.988+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15541a9b10 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.989+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1554063f50 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.992+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f154000c7e0 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.993+0000 7f1551600700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f153c0b2910 0x7f153c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1558a00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f153c0b2910 0x7f153c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f15400e2a00 con 0x7f1554076450 2024-05-10T21:06:04.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1551600700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] conn(0x7f153c0bc3a0 0x7f153c0be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f153c0bef60 con 0x7f153c0bc3a0 2024-05-10T21:06:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f15400e2db0 con 0x7f1554076450 2024-05-10T21:06:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.994+0000 7f1559400700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] conn(0x7f153c0bc3a0 0x7f153c0be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.995+0000 7f1558a00700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f153c0b2910 0x7f153c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15480098e0 tx=0x7f1548005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.995+0000 7f1559400700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] conn(0x7f153c0bc3a0 0x7f153c0be890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:03.996+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== osd.3 v2:172.21.15.50:6810/1367886270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f153c0bef60 con 0x7f153c0bc3a0 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.014+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 --> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15540643d0 con 0x7f153c0bc3a0 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.015+0000 7f1551600700 1 -- 172.21.15.50:0/190721897 <== osd.3 v2:172.21.15.50:6810/1367886270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f15540643d0 con 0x7f153c0bc3a0 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.015+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] conn(0x7f153c0bc3a0 msgr2=0x7f153c0be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.015+0000 7f155b400700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] conn(0x7f153c0bc3a0 0x7f153c0be890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f153c0b2910 msgr2=0x7f153c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f153c0b2910 0x7f153c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15480098e0 tx=0x7f1548005960 comp rx=0 tx=0).stop 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 msgr2=0x7f155419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 --2- 172.21.15.50:0/190721897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1554076450 0x7f155419d5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f154000e970 tx=0x7f154000ec80 comp rx=0 tx=0).stop 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f1559400700 1 -- 172.21.15.50:0/190721897 reap_dead start 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 shutdown_connections 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 >> 172.21.15.50:0/190721897 conn(0x7f1554100040 msgr2=0x7f1554100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:04.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 shutdown_connections 2024-05-10T21:06:04.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:04.016+0000 7f155b400700 1 -- 172.21.15.50:0/190721897 wait complete. 2024-05-10T21:06:04.853 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:04 smithi050 ceph-mon[29546]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:05.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:04 smithi092 ceph-mon[34415]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:05.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:04 smithi142 ceph-mon[34846]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:05.507 INFO:teuthology.orchestra.run.smithi050.stdout:107374182482 2024-05-10T21:06:05.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 -- 172.21.15.50:0/3336194287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 msgr2=0x7f8bd0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3336194287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd0104a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00099c0 tx=0x7f8bc0009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 -- 172.21.15.50:0/3336194287 shutdown_connections 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3336194287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0109890 0x7f8bd01111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3336194287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd0104a20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3336194287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 0x7f8bd0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 -- 172.21.15.50:0/3336194287 >> 172.21.15.50:0/3336194287 conn(0x7f8bd007af70 msgr2=0x7f8bd007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:05.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 -- 172.21.15.50:0/3336194287 shutdown_connections 2024-05-10T21:06:05.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.772+0000 7f8bd6200700 1 -- 172.21.15.50:0/3336194287 wait complete. 2024-05-10T21:06:05.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.774+0000 7f8bd6200700 1 Processor -- start 2024-05-10T21:06:05.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.774+0000 7f8bd6200700 1 -- start start 2024-05-10T21:06:05.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd00728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 0x7f8bd0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0109890 0x7f8bd0073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0114510 con 0x7f8bd0104610 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0114390 con 0x7f8bd0108f40 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.775+0000 7f8bd6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0114690 con 0x7f8bd0109890 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd00728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd00728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37936/0 (socket says 172.21.15.50:37936) 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcf400700 1 -- 172.21.15.50:0/3787609375 learned_addr learned my addr 172.21.15.50:0/3787609375 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 0x7f8bd0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcfe00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0109890 0x7f8bd0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 -- 172.21.15.50:0/3787609375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0109890 msgr2=0x7f8bd0073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0109890 0x7f8bd0073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 -- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 msgr2=0x7f8bd00728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd00728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcea00700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd01b1a30 con 0x7f8bd0108f40 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.776+0000 7f8bcf400700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd0104610 0x7f8bd00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.777+0000 7f8bcea00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 0x7f8bd0072e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e9c0 tx=0x7f8bbc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.777+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00cbd0 con 0x7f8bd0108f40 2024-05-10T21:06:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.777+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bbc00cd70 con 0x7f8bd0108f40 2024-05-10T21:06:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.777+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc018680 con 0x7f8bd0108f40 2024-05-10T21:06:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.777+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd01b1cc0 con 0x7f8bd0108f40 2024-05-10T21:06:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.778+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01b21b0 con 0x7f8bd0108f40 2024-05-10T21:06:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.779+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bd0063f50 con 0x7f8bd0108f40 2024-05-10T21:06:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.783+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8bbc010430 con 0x7f8bd0108f40 2024-05-10T21:06:05.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.785+0000 7f8bc7e00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb40b2990 0x7f8bb40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:05.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.785+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f8bbc0e31c0 con 0x7f8bd0108f40 2024-05-10T21:06:05.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.785+0000 7f8bc7e00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] conn(0x7f8bb40bc420 0x7f8bb40be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:05.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.786+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bbc016ec0 con 0x7f8bb40bc420 2024-05-10T21:06:05.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.786+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc0e3570 con 0x7f8bd0108f40 2024-05-10T21:06:05.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.786+0000 7f8bcf400700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb40b2990 0x7f8bb40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.787+0000 7f8bcf400700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb40b2990 0x7f8bb40b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009990 tx=0x7f8bc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:05.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.787+0000 7f8bcfe00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] conn(0x7f8bb40bc420 0x7f8bb40be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:05.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.788+0000 7f8bcfe00700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] conn(0x7f8bb40bc420 0x7f8bb40be910 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:05.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.789+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== osd.9 v2:172.21.15.50:6826/2893617399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bbc016ec0 con 0x7f8bb40bc420 2024-05-10T21:06:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.809+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 --> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bd00643d0 con 0x7f8bb40bc420 2024-05-10T21:06:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.810+0000 7f8bc7e00700 1 -- 172.21.15.50:0/3787609375 <== osd.9 v2:172.21.15.50:6826/2893617399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8bd00643d0 con 0x7f8bb40bc420 2024-05-10T21:06:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.810+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] conn(0x7f8bb40bc420 msgr2=0x7f8bb40be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] conn(0x7f8bb40bc420 0x7f8bb40be910 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb40b2990 msgr2=0x7f8bb40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb40b2990 0x7f8bb40b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009990 tx=0x7f8bc001a040 comp rx=0 tx=0).stop 2024-05-10T21:06:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 msgr2=0x7f8bd0072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 --2- 172.21.15.50:0/3787609375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd0108f40 0x7f8bd0072e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e9c0 tx=0x7f8bbc00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:06:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bcfe00700 1 -- 172.21.15.50:0/3787609375 reap_dead start 2024-05-10T21:06:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 shutdown_connections 2024-05-10T21:06:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 >> 172.21.15.50:0/3787609375 conn(0x7f8bd007af70 msgr2=0x7f8bd01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 shutdown_connections 2024-05-10T21:06:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:05.811+0000 7f8bd6200700 1 -- 172.21.15.50:0/3787609375 wait complete. 2024-05-10T21:06:06.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:06 smithi050 ceph-mon[29546]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:07.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:06 smithi092 ceph-mon[34415]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:07.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:06 smithi142 ceph-mon[34846]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:07.140 INFO:teuthology.orchestra.run.smithi050.stdout:231928234039 2024-05-10T21:06:07.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 -- 172.21.15.50:0/2706681632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c1045f0 msgr2=0x7f995c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 --2- 172.21.15.50:0/2706681632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c1045f0 0x7f995c104a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f994c0099c0 tx=0x7f994c009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 -- 172.21.15.50:0/2706681632 shutdown_connections 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 --2- 172.21.15.50:0/2706681632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c109870 0x7f995c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 --2- 172.21.15.50:0/2706681632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c1045f0 0x7f995c104a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 --2- 172.21.15.50:0/2706681632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c108f20 0x7f995c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 -- 172.21.15.50:0/2706681632 >> 172.21.15.50:0/2706681632 conn(0x7f995c100040 msgr2=0x7f995c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 -- 172.21.15.50:0/2706681632 shutdown_connections 2024-05-10T21:06:07.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.200+0000 7f9960e00700 1 -- 172.21.15.50:0/2706681632 wait complete. 2024-05-10T21:06:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.202+0000 7f9960e00700 1 Processor -- start 2024-05-10T21:06:07.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.202+0000 7f9960e00700 1 -- start start 2024-05-10T21:06:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 0x7f995c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c108f20 0x7f995c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c109870 0x7f995c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1144a0 con 0x7f995c109870 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f995c114320 con 0x7f995c108f20 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9960e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f995c114620 con 0x7f995c1045f0 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9959e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 0x7f995c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9959e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 0x7f995c19d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60212/0 (socket says 172.21.15.50:60212) 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9959e00700 1 -- 172.21.15.50:0/151795186 learned_addr learned my addr 172.21.15.50:0/151795186 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f995a800700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c109870 0x7f995c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.203+0000 7f9959400700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c108f20 0x7f995c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 -- 172.21.15.50:0/151795186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c108f20 msgr2=0x7f995c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f995c108f20 0x7f995c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 -- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c109870 msgr2=0x7f995c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f995c109870 0x7f995c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c1a1f00 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9959e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 0x7f995c19d030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.204+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c026070 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.205+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c00f460 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.205+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c1a2190 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.205+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c009e60 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.205+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1a2670 con 0x7f995c1045f0 2024-05-10T21:06:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.206+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f995c1107f0 con 0x7f995c1045f0 2024-05-10T21:06:07.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.209+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f994c029070 con 0x7f995c1045f0 2024-05-10T21:06:07.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.210+0000 7f9952a00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99400b2910 0x7f99400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.210+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f994c0e3290 con 0x7f995c1045f0 2024-05-10T21:06:07.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.211+0000 7f9959400700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99400b2910 0x7f99400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.211+0000 7f9952a00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] conn(0x7f99400bc3a0 0x7f99400be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.211+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f994c00eea0 con 0x7f99400bc3a0 2024-05-10T21:06:07.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.211+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f994c0e3640 con 0x7f995c1045f0 2024-05-10T21:06:07.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.211+0000 7f995a800700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] conn(0x7f99400bc3a0 0x7f99400be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.212+0000 7f9959400700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99400b2910 0x7f99400b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f995c108df0 tx=0x7f99480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.212+0000 7f995a800700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] conn(0x7f99400bc3a0 0x7f99400be890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.213+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== osd.5 v2:172.21.15.142:6808/3719494232 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f994c00eea0 con 0x7f99400bc3a0 2024-05-10T21:06:07.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.232+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 --> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f995c105aa0 con 0x7f99400bc3a0 2024-05-10T21:06:07.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.232+0000 7f9952a00700 1 -- 172.21.15.50:0/151795186 <== osd.5 v2:172.21.15.142:6808/3719494232 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f995c105aa0 con 0x7f99400bc3a0 2024-05-10T21:06:07.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] conn(0x7f99400bc3a0 msgr2=0x7f99400be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] conn(0x7f99400bc3a0 0x7f99400be890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99400b2910 msgr2=0x7f99400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f99400b2910 0x7f99400b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f995c108df0 tx=0x7f99480073d0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 msgr2=0x7f995c19d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f9960e00700 1 --2- 172.21.15.50:0/151795186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f995c1045f0 0x7f995c19d030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c0055f0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.233+0000 7f995a800700 1 -- 172.21.15.50:0/151795186 reap_dead start 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.234+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 shutdown_connections 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.234+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 >> 172.21.15.50:0/151795186 conn(0x7f995c100040 msgr2=0x7f995c1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.234+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 shutdown_connections 2024-05-10T21:06:07.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:07.234+0000 7f9960e00700 1 -- 172.21.15.50:0/151795186 wait complete. 2024-05-10T21:06:07.296 INFO:teuthology.orchestra.run.smithi050.stdout:283467841578 2024-05-10T21:06:07.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T21:06:07.449 INFO:teuthology.orchestra.run.smithi050.stdout:180388626496 2024-05-10T21:06:07.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 -- 172.21.15.50:0/2216053641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 msgr2=0x7fee381111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 --2- 172.21.15.50:0/2216053641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 0x7fee381111d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b0d0 tx=0x7fee3000b3e0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 -- 172.21.15.50:0/2216053641 shutdown_connections 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 --2- 172.21.15.50:0/2216053641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 0x7fee381111d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 --2- 172.21.15.50:0/2216053641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee38104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 --2- 172.21.15.50:0/2216053641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 0x7fee38109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 -- 172.21.15.50:0/2216053641 >> 172.21.15.50:0/2216053641 conn(0x7fee38100040 msgr2=0x7fee38102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 -- 172.21.15.50:0/2216053641 shutdown_connections 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.588+0000 7fee3ca00700 1 -- 172.21.15.50:0/2216053641 wait complete. 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.590+0000 7fee3ca00700 1 Processor -- start 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.590+0000 7fee3ca00700 1 -- start start 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee3819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 0x7fee3819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 0x7fee381a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee381144f0 con 0x7fee38109870 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee38114370 con 0x7fee38108f20 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee3ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee38114670 con 0x7fee381045f0 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.591+0000 7fee35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee3819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 0x7fee3819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee3819d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60196/0 (socket says 172.21.15.50:60196) 2024-05-10T21:06:07.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 -- 172.21.15.50:0/3979777423 learned_addr learned my addr 172.21.15.50:0/3979777423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee36800700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 0x7fee381a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 -- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 msgr2=0x7fee3819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 0x7fee3819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 -- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 msgr2=0x7fee381a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee38109870 0x7fee381a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35e00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee381a1f10 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.592+0000 7fee35400700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee38108f20 0x7fee3819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee35e00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee3819d040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb30 tx=0x7fee2800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800cbd0 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280044d0 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee381a2200 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28003e30 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.593+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381a9b20 con 0x7fee381045f0 2024-05-10T21:06:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.594+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee381080e0 con 0x7fee381045f0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.597+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee28004000 con 0x7fee381045f0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee2ea00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee1c0b2910 0x7fee1c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fee280e2d70 con 0x7fee381045f0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee35400700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee1c0b2910 0x7fee1c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee2ea00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] conn(0x7fee1c0bc3a0 0x7fee1c0be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee1c0bef60 con 0x7fee1c0bc3a0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.599+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fee280e3120 con 0x7fee381045f0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.600+0000 7fee36800700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] conn(0x7fee1c0bc3a0 0x7fee1c0be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.600+0000 7fee35400700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee1c0b2910 0x7fee1c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee24009680 tx=0x7fee24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.600+0000 7fee36800700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] conn(0x7fee1c0bc3a0 0x7fee1c0be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.601+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== osd.7 v2:172.21.15.92:6816/477438803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee1c0bef60 con 0x7fee1c0bc3a0 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.620+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 --> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee38105aa0 con 0x7fee1c0bc3a0 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.621+0000 7fee2ea00700 1 -- 172.21.15.50:0/3979777423 <== osd.7 v2:172.21.15.92:6816/477438803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fee38105aa0 con 0x7fee1c0bc3a0 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] conn(0x7fee1c0bc3a0 msgr2=0x7fee1c0be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] conn(0x7fee1c0bc3a0 0x7fee1c0be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee1c0b2910 msgr2=0x7fee1c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee1c0b2910 0x7fee1c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee24009680 tx=0x7fee24009260 comp rx=0 tx=0).stop 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 msgr2=0x7fee3819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 --2- 172.21.15.50:0/3979777423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee381045f0 0x7fee3819d040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb30 tx=0x7fee2800ee40 comp rx=0 tx=0).stop 2024-05-10T21:06:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee36800700 1 -- 172.21.15.50:0/3979777423 reap_dead start 2024-05-10T21:06:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 shutdown_connections 2024-05-10T21:06:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 >> 172.21.15.50:0/3979777423 conn(0x7fee38100040 msgr2=0x7fee38110e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.622+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 shutdown_connections 2024-05-10T21:06:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:06.623+0000 7fee3ca00700 1 -- 172.21.15.50:0/3979777423 wait complete. 2024-05-10T21:06:08.245 INFO:teuthology.orchestra.run.smithi050.stdout:257698037809 2024-05-10T21:06:08.245 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T21:06:09.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:08 smithi092 ceph-mon[34415]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-10T21:06:09.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:08 smithi142 ceph-mon[34846]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-10T21:06:09.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:08 smithi050 ceph-mon[29546]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-10T21:06:09.403 INFO:teuthology.orchestra.run.smithi050.stdout:309237645351 2024-05-10T21:06:09.403 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 -- 172.21.15.50:0/741272614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c109680 msgr2=0x7f683c110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 --2- 172.21.15.50:0/741272614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c109680 0x7f683c110fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f683400b0d0 tx=0x7f683400b3e0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 -- 172.21.15.50:0/741272614 shutdown_connections 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 --2- 172.21.15.50:0/741272614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c109680 0x7f683c110fe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 --2- 172.21.15.50:0/741272614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f683c104400 0x7f683c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 --2- 172.21.15.50:0/741272614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c108d30 0x7f683c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 -- 172.21.15.50:0/741272614 >> 172.21.15.50:0/741272614 conn(0x7f683c079d10 msgr2=0x7f683c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 -- 172.21.15.50:0/741272614 shutdown_connections 2024-05-10T21:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.710+0000 7f6841600700 1 -- 172.21.15.50:0/741272614 wait complete. 2024-05-10T21:06:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 Processor -- start 2024-05-10T21:06:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 -- start start 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 0x7f683c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c108d30 0x7f683c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f683c109680 0x7f683c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1142b0 con 0x7f683c108d30 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f683c114130 con 0x7f683c104400 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.712+0000 7f6841600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f683c114430 con 0x7f683c109680 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 0x7f683c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c108d30 0x7f683c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 0x7f683c1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57176/0 (socket says 172.21.15.50:57176) 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 -- 172.21.15.50:0/2571203729 learned_addr learned my addr 172.21.15.50:0/2571203729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f6840c00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f683c109680 0x7f683c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 -- 172.21.15.50:0/2571203729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f683c109680 msgr2=0x7f683c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f683c109680 0x7f683c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 -- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c108d30 msgr2=0x7f683c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c108d30 0x7f683c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683be00700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c19b160 con 0x7f683c104400 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.713+0000 7f683b400700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f683c108d30 0x7f683c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f683be00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 0x7f683c1a0890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9a0 tx=0x7f682c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c00ca10 con 0x7f683c104400 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c00cbb0 con 0x7f683c104400 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c19b450 con 0x7f683c104400 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c018680 con 0x7f683c104400 2024-05-10T21:06:09.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.714+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a7720 con 0x7f683c104400 2024-05-10T21:06:09.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.715+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f681c001030 con 0x7f683c104400 2024-05-10T21:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.719+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f682c010430 con 0x7f683c104400 2024-05-10T21:06:09.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.721+0000 7f6838c00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68200b2990 0x7f68200b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.721+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f682c0e2410 con 0x7f683c104400 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.721+0000 7f683b400700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68200b2990 0x7f68200b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.721+0000 7f6838c00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] conn(0x7f68200bc420 0x7f68200be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.722+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68200befe0 con 0x7f68200bc420 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.722+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f682c0e9050 con 0x7f683c104400 2024-05-10T21:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.722+0000 7f6840c00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] conn(0x7f68200bc420 0x7f68200be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.722+0000 7f683b400700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68200b2990 0x7f68200b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6828009680 tx=0x7f6828009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.723+0000 7f6840c00700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] conn(0x7f68200bc420 0x7f68200be910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.724+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== osd.0 v2:172.21.15.50:6802/2398884233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68200befe0 con 0x7f68200bc420 2024-05-10T21:06:09.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.742+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 --> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f681c002e60 con 0x7f68200bc420 2024-05-10T21:06:09.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.743+0000 7f6838c00700 1 -- 172.21.15.50:0/2571203729 <== osd.0 v2:172.21.15.50:6802/2398884233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f681c002e60 con 0x7f68200bc420 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.743+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] conn(0x7f68200bc420 msgr2=0x7f68200be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.743+0000 7f6841600700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] conn(0x7f68200bc420 0x7f68200be910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68200b2990 msgr2=0x7f68200b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68200b2990 0x7f68200b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6828009680 tx=0x7f6828009260 comp rx=0 tx=0).stop 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 msgr2=0x7f683c1a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 --2- 172.21.15.50:0/2571203729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f683c104400 0x7f683c1a0890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9a0 tx=0x7f682c00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6840c00700 1 -- 172.21.15.50:0/2571203729 reap_dead start 2024-05-10T21:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 shutdown_connections 2024-05-10T21:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 >> 172.21.15.50:0/2571203729 conn(0x7f683c079d10 msgr2=0x7f683c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 shutdown_connections 2024-05-10T21:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:09.744+0000 7f6841600700 1 -- 172.21.15.50:0/2571203729 wait complete. 2024-05-10T21:06:10.202 INFO:teuthology.orchestra.run.smithi050.stdout:55834574944 2024-05-10T21:06:10.203 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T21:06:10.282 INFO:teuthology.orchestra.run.smithi050.stdout:154618822729 2024-05-10T21:06:10.283 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T21:06:10.755 INFO:teuthology.orchestra.run.smithi050.stdout:34359738470 2024-05-10T21:06:10.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T21:06:11.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:10 smithi092 ceph-mon[34415]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:11.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:10 smithi142 ceph-mon[34846]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:11.162 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:10 smithi050 ceph-mon[29546]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:11.328 INFO:teuthology.orchestra.run.smithi050.stdout:206158430271 2024-05-10T21:06:11.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T21:06:11.387 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.387 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.393 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.395 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.412 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.415 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.506 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:11.556 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:12.034 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:13.032 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:12 smithi050 ceph-mon[29546]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:13.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:12 smithi092 ceph-mon[34415]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:13.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:12 smithi142 ceph-mon[34846]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:13.598 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:14.992 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:14 smithi050 ceph-mon[29546]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:15.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:14 smithi092 ceph-mon[34415]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:15.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:14 smithi142 ceph-mon[34846]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:17.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:16 smithi050 ceph-mon[29546]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:17.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:16 smithi092 ceph-mon[34415]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:17.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:16 smithi142 ceph-mon[34846]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:18.998 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:18 smithi050 ceph-mon[29546]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 -- 172.21.15.50:0/976458542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c109860 msgr2=0x7f0d9c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 --2- 172.21.15.50:0/976458542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c109860 0x7f0d9c1111c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009f90 tx=0x7f0d9400b330 comp rx=0 tx=0).stop 2024-05-10T21:06:19.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 -- 172.21.15.50:0/976458542 shutdown_connections 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 --2- 172.21.15.50:0/976458542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c109860 0x7f0d9c1111c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 --2- 172.21.15.50:0/976458542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 0x7f0d9c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 --2- 172.21.15.50:0/976458542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c108f10 0x7f0d9c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 -- 172.21.15.50:0/976458542 >> 172.21.15.50:0/976458542 conn(0x7f0d9c0ffe70 msgr2=0x7f0d9c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 -- 172.21.15.50:0/976458542 shutdown_connections 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.722+0000 7f0da4600700 1 -- 172.21.15.50:0/976458542 wait complete. 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.723+0000 7f0da4600700 1 Processor -- start 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 -- start start 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 0x7f0d9c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c108f10 0x7f0d9c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:19.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1144e0 con 0x7f0d9c1045e0 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c114360 con 0x7f0d9c108f10 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.724+0000 7f0da4600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c114660 con 0x7f0d9c109860 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da1c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 0x7f0d9c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35866/0 (socket says 172.21.15.50:35866) 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 -- 172.21.15.50:0/3038975409 learned_addr learned my addr 172.21.15.50:0/3038975409 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da1200700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c108f10 0x7f0d9c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 -- 172.21.15.50:0/3038975409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c108f10 msgr2=0x7f0d9c19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c108f10 0x7f0d9c19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 -- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 msgr2=0x7f0d9c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 0x7f0d9c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 -- 172.21.15.50:0/3038975409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a1ec0 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.725+0000 7f0da2600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009c10 tx=0x7f0d9400cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.732+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94021070 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.732+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d9401d450 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.732+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a2150 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.732+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d940052a0 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.733+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a9ad0 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.734+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c105a90 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.736+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0d94024070 con 0x7f0d9c109860 2024-05-10T21:06:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.738+0000 7f0d9a000700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 0x7f0d840b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.738+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0d9400f020 con 0x7f0d9c109860 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.738+0000 7f0da1c00700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 0x7f0d840b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.739+0000 7f0da1c00700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 0x7f0d840b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009c60 tx=0x7f0d90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.743+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d9402b020 con 0x7f0d9c109860 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.996+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0d9c1a9ca0 con 0x7f0d9c109860 2024-05-10T21:06:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:18.997+0000 7f0d9a000700 1 -- 172.21.15.50:0/3038975409 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0d940ab350 con 0x7f0d9c109860 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stdout:309237645355 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.000+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 msgr2=0x7f0d840b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 0x7f0d840b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009c60 tx=0x7f0d90009260 comp rx=0 tx=0).stop 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 msgr2=0x7f0d9c1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009c10 tx=0x7f0d9400cca0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 shutdown_connections 2024-05-10T21:06:19.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0d840b2990 0x7f0d840b4ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c109860 0x7f0d9c1a1820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d9c1045e0 0x7f0d9c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 --2- 172.21.15.50:0/3038975409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d9c108f10 0x7f0d9c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 >> 172.21.15.50:0/3038975409 conn(0x7f0d9c0ffe70 msgr2=0x7f0d9c10fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 shutdown_connections 2024-05-10T21:06:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:19.001+0000 7f0da4600700 1 -- 172.21.15.50:0/3038975409 wait complete. 2024-05-10T21:06:19.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:18 smithi092 ceph-mon[34415]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:19.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:18 smithi142 ceph-mon[34846]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:20.022 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3038975409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T21:06:20.114 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3038975409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T21:06:20.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3038975409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T21:06:20.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:20 smithi050 ceph-mon[29546]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:20.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:06:20.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1459568181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T21:06:20.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:20.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:20 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:20 smithi092 ceph-mon[34415]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:06:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1459568181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T21:06:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:21.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:20 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:21.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:20 smithi142 ceph-mon[34846]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:21.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:06:21.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1459568181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T21:06:21.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:21.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:20 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:21.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 -- 172.21.15.50:0/2025624668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 msgr2=0x7f87ec1106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 --2- 172.21.15.50:0/2025624668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec1106b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009990 tx=0x7f87d8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 -- 172.21.15.50:0/2025624668 shutdown_connections 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 --2- 172.21.15.50:0/2025624668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 0x7f87ec113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 --2- 172.21.15.50:0/2025624668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 0x7f87ec077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 --2- 172.21.15.50:0/2025624668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec1106b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 -- 172.21.15.50:0/2025624668 >> 172.21.15.50:0/2025624668 conn(0x7f87ec0ffed0 msgr2=0x7f87ec102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 -- 172.21.15.50:0/2025624668 shutdown_connections 2024-05-10T21:06:21.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.357+0000 7f87f3400700 1 -- 172.21.15.50:0/2025624668 wait complete. 2024-05-10T21:06:21.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 Processor -- start 2024-05-10T21:06:21.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 -- start start 2024-05-10T21:06:21.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 0x7f87ec19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:21.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:21.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 0x7f87ec1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec116700 con 0x7f87ec076c10 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec116580 con 0x7f87ec110bf0 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.359+0000 7f87f3400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec116880 con 0x7f87ec077560 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35900/0 (socket says 172.21.15.50:35900) 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 -- 172.21.15.50:0/808989904 learned_addr learned my addr 172.21.15.50:0/808989904 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87f0a00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 0x7f87ec19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87f1400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 0x7f87ec1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 -- 172.21.15.50:0/808989904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 msgr2=0x7f87ec1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 0x7f87ec1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 -- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 msgr2=0x7f87ec19d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 0x7f87ec19d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.360+0000 7f87ebe00700 1 -- 172.21.15.50:0/808989904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec1a1f90 con 0x7f87ec077560 2024-05-10T21:06:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87ebe00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87d8000c00 tx=0x7f87d80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8027070 con 0x7f87ec077560 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d8022470 con 0x7f87ec077560 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec1a2220 con 0x7f87ec077560 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8017650 con 0x7f87ec077560 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.361+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1a9ba0 con 0x7f87ec077560 2024-05-10T21:06:21.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.362+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ec078350 con 0x7f87ec077560 2024-05-10T21:06:21.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.366+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f87d802a070 con 0x7f87ec077560 2024-05-10T21:06:21.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.367+0000 7f87e9600700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 0x7f87d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:21.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.367+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f87d80e3390 con 0x7f87ec077560 2024-05-10T21:06:21.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.367+0000 7f87f0a00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 0x7f87d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:21.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.368+0000 7f87f0a00700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 0x7f87d40b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009c60 tx=0x7f87e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:21.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.369+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87d8013070 con 0x7f87ec077560 2024-05-10T21:06:21.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.626+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f87ec19e420 con 0x7f87ec077560 2024-05-10T21:06:21.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.628+0000 7f87e9600700 1 -- 172.21.15.50:0/808989904 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f87d80aafb0 con 0x7f87ec077560 2024-05-10T21:06:21.630 INFO:teuthology.orchestra.run.smithi050.stdout:257698037812 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 msgr2=0x7f87d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 0x7f87d40b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009c60 tx=0x7f87e0009260 comp rx=0 tx=0).stop 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 msgr2=0x7f87ec19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87d8000c00 tx=0x7f87d80057c0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 shutdown_connections 2024-05-10T21:06:21.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f87d40b2990 0x7f87d40b4ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec110bf0 0x7f87ec1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87ec076c10 0x7f87ec19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 --2- 172.21.15.50:0/808989904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87ec077560 0x7f87ec19d6b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 >> 172.21.15.50:0/808989904 conn(0x7f87ec0ffed0 msgr2=0x7f87ec1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 shutdown_connections 2024-05-10T21:06:21.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:21.631+0000 7f87f3400700 1 -- 172.21.15.50:0/808989904 wait complete. 2024-05-10T21:06:22.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 -- 172.21.15.50:0/193381986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105310 msgr2=0x7f95a4105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 --2- 172.21.15.50:0/193381986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105310 0x7f95a4105720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9594009990 tx=0x7f9594009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 -- 172.21.15.50:0/193381986 shutdown_connections 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 --2- 172.21.15.50:0/193381986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a410e740 0x7f95a4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 --2- 172.21.15.50:0/193381986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105310 0x7f95a4105720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 --2- 172.21.15.50:0/193381986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a4105c60 0x7f95a410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 -- 172.21.15.50:0/193381986 >> 172.21.15.50:0/193381986 conn(0x7f95a4078ac0 msgr2=0x7f95a4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 -- 172.21.15.50:0/193381986 shutdown_connections 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.289+0000 7f95a3e00700 1 -- 172.21.15.50:0/193381986 wait complete. 2024-05-10T21:06:22.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 Processor -- start 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 -- start start 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 0x7f95a4106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410e740 0x7f95a4106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41142a0 con 0x7f95a4105310 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114120 con 0x7f95a410e740 2024-05-10T21:06:22.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.291+0000 7f95a3e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114420 con 0x7f95a4105c60 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 0x7f95a4106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35918/0 (socket says 172.21.15.50:35918) 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 -- 172.21.15.50:0/129318876 learned_addr learned my addr 172.21.15.50:0/129318876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a3400700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410e740 0x7f95a4106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 -- 172.21.15.50:0/129318876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410e740 msgr2=0x7f95a4106ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410e740 0x7f95a4106ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 -- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 msgr2=0x7f95a4106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 0x7f95a4106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.292+0000 7f95a2000700 1 -- 172.21.15.50:0/129318876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41ad3f0 con 0x7f95a4105c60 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f95a2a00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 0x7f95a4106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f95a2000700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f959000e9a0 tx=0x7f959000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959000ca10 con 0x7f95a4105c60 2024-05-10T21:06:22.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959000cbb0 con 0x7f95a4105c60 2024-05-10T21:06:22.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41ad680 con 0x7f95a4105c60 2024-05-10T21:06:22.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590018680 con 0x7f95a4105c60 2024-05-10T21:06:22.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.293+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41adc10 con 0x7f95a4105c60 2024-05-10T21:06:22.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.295+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a406be60 con 0x7f95a4105c60 2024-05-10T21:06:22.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.297+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9590010430 con 0x7f95a4105c60 2024-05-10T21:06:22.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.299+0000 7f959b400700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 0x7f95880b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.299+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f95900e24c0 con 0x7f95a4105c60 2024-05-10T21:06:22.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.299+0000 7f95a2a00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 0x7f95880b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.300+0000 7f95a2a00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 0x7f95880b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f959400b2e0 tx=0x7f9594005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:22.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.303+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95900aa050 con 0x7f95a4105c60 2024-05-10T21:06:22.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 -- 172.21.15.50:0/2920180552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 msgr2=0x7f4964104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 --2- 172.21.15.50:0/2920180552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f4964104810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49540099c0 tx=0x7f4954009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 -- 172.21.15.50:0/2920180552 shutdown_connections 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 --2- 172.21.15.50:0/2920180552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 0x7f4964110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 --2- 172.21.15.50:0/2920180552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f4964104810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 --2- 172.21.15.50:0/2920180552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 0x7f4964109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 -- 172.21.15.50:0/2920180552 >> 172.21.15.50:0/2920180552 conn(0x7f4964079d10 msgr2=0x7f496407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 -- 172.21.15.50:0/2920180552 shutdown_connections 2024-05-10T21:06:22.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.508+0000 7f4968a00700 1 -- 172.21.15.50:0/2920180552 wait complete. 2024-05-10T21:06:22.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.509+0000 7f4968a00700 1 Processor -- start 2024-05-10T21:06:22.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 -- start start 2024-05-10T21:06:22.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 0x7f49641a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 0x7f496419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49641142b0 con 0x7f4964109680 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4964114130 con 0x7f4964108d30 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4968a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4964114430 con 0x7f4964104400 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4963e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 0x7f496419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.510+0000 7f4963400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35936/0 (socket says 172.21.15.50:35936) 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 -- 172.21.15.50:0/3376687672 learned_addr learned my addr 172.21.15.50:0/3376687672 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4962a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 0x7f49641a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 -- 172.21.15.50:0/3376687672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 msgr2=0x7f49641a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 0x7f49641a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 -- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 msgr2=0x7f496419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 0x7f496419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 -- 172.21.15.50:0/3376687672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496419b180 con 0x7f4964104400 2024-05-10T21:06:22.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.511+0000 7f4963400700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4954006010 tx=0x7f4954004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:22.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.512+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954026070 con 0x7f4964104400 2024-05-10T21:06:22.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.512+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4954004630 con 0x7f4964104400 2024-05-10T21:06:22.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.512+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400fb20 con 0x7f4964104400 2024-05-10T21:06:22.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.512+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496419b410 con 0x7f4964104400 2024-05-10T21:06:22.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.512+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496419b8a0 con 0x7f4964104400 2024-05-10T21:06:22.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.513+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49641058b0 con 0x7f4964104400 2024-05-10T21:06:22.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.516+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f49540037c0 con 0x7f4964104400 2024-05-10T21:06:22.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.517+0000 7f495be00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 0x7f49480b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:22.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.517+0000 7f4962a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 0x7f49480b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:22.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.517+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f49540e2c80 con 0x7f4964104400 2024-05-10T21:06:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.518+0000 7f4962a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 0x7f49480b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f496407a240 tx=0x7f4950009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:22.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.521+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4954013070 con 0x7f4964104400 2024-05-10T21:06:22.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.558+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f95a41ade70 con 0x7f95a4105c60 2024-05-10T21:06:22.560 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.560 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.560 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.560 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.561 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:22.561 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:22 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/808989904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T21:06:22.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.558+0000 7f959b400700 1 -- 172.21.15.50:0/129318876 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f95900b20d0 con 0x7f95a4105c60 2024-05-10T21:06:22.561 INFO:teuthology.orchestra.run.smithi050.stdout:180388626501 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.562+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 msgr2=0x7f95880b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.562+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 0x7f95880b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f959400b2e0 tx=0x7f9594005fd0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 msgr2=0x7f95a4106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f959000e9a0 tx=0x7f959000ecb0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 shutdown_connections 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95880b2990 0x7f95880b4ea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95a410e740 0x7f95a4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95a4105310 0x7f95a4106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 --2- 172.21.15.50:0/129318876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a4105c60 0x7f95a4106760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 >> 172.21.15.50:0/129318876 conn(0x7f95a4078ac0 msgr2=0x7f95a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:22.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 shutdown_connections 2024-05-10T21:06:22.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.563+0000 7f95a3e00700 1 -- 172.21.15.50:0/129318876 wait complete. 2024-05-10T21:06:22.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.787+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f496402e5f0 con 0x7f4964104400 2024-05-10T21:06:22.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.787+0000 7f495be00700 1 -- 172.21.15.50:0/3376687672 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49540aa8a0 con 0x7f4964104400 2024-05-10T21:06:22.790 INFO:teuthology.orchestra.run.smithi050.stdout:231928234043 2024-05-10T21:06:22.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 msgr2=0x7f49480b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 0x7f49480b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f496407a240 tx=0x7f4950009260 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 msgr2=0x7f49641a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4954006010 tx=0x7f4954004270 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 shutdown_connections 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49480b2990 0x7f49480b4ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4964109680 0x7f496419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4964104400 0x7f49641a0860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 --2- 172.21.15.50:0/3376687672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4964108d30 0x7f49641a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.790+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 >> 172.21.15.50:0/3376687672 conn(0x7f4964079d10 msgr2=0x7f4964102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.791+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 shutdown_connections 2024-05-10T21:06:22.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:22.791+0000 7f4968a00700 1 -- 172.21.15.50:0/3376687672 wait complete. 2024-05-10T21:06:22.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:22.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:22 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/808989904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:22.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:22 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/808989904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 -- 172.21.15.50:0/3023566551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e81025a0 msgr2=0x7f58e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 --2- 172.21.15.50:0/3023566551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e81025a0 0x7f58e81029b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009990 tx=0x7f58d4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 -- 172.21.15.50:0/3023566551 shutdown_connections 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 --2- 172.21.15.50:0/3023566551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e8102ef0 0x7f58e8110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 --2- 172.21.15.50:0/3023566551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e8109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 --2- 172.21.15.50:0/3023566551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e81025a0 0x7f58e81029b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.111+0000 7f58e7200700 1 -- 172.21.15.50:0/3023566551 >> 172.21.15.50:0/3023566551 conn(0x7f58e80783b0 msgr2=0x7f58e80787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.112+0000 7f58e7200700 1 -- 172.21.15.50:0/3023566551 shutdown_connections 2024-05-10T21:06:23.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.112+0000 7f58e7200700 1 -- 172.21.15.50:0/3023566551 wait complete. 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.113+0000 7f58e7200700 1 Processor -- start 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 -- start start 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e8102ef0 0x7f58e819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58e81143b0 con 0x7f58e8109110 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8114230 con 0x7f58e8102ef0 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.114+0000 7f58e7200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8114530 con 0x7f58e81025a0 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35886/0 (socket says 172.21.15.50:35886) 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 -- 172.21.15.50:0/1459568181 learned_addr learned my addr 172.21.15.50:0/1459568181 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5400700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e8102ef0 0x7f58e819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:23.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e6800700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 -- 172.21.15.50:0/1459568181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e8102ef0 msgr2=0x7f58e819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e8102ef0 0x7f58e819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 -- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 msgr2=0x7f58e81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 -- 172.21.15.50:0/1459568181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e81a1db0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e6800700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.115+0000 7f58e5e00700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e940 tx=0x7f58d800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.116+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d800cbd0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.116+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e81a20a0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.116+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d800ee90 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.116+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8003dc0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.116+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81a99c0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.117+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e81058c0 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.119+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f58d8003f60 con 0x7f58e81025a0 2024-05-10T21:06:23.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.121+0000 7f58dea00700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 0x7f58cc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.121+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f58d80e25e0 con 0x7f58e81025a0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.121+0000 7f58e5400700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 0x7f58cc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.122+0000 7f58e5400700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 0x7f58cc0b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b2e0 tx=0x7f58d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.125+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58d80aa170 con 0x7f58e81025a0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.389+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f58e81a9b90 con 0x7f58e81025a0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.389+0000 7f58dea00700 1 -- 172.21.15.50:0/1459568181 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f58d80a9a70 con 0x7f58e81025a0 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stdout:283467841582 2024-05-10T21:06:23.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 msgr2=0x7f58cc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 0x7f58cc0b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b2e0 tx=0x7f58d4005960 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 msgr2=0x7f58e819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58d800e940 tx=0x7f58d800ec50 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 shutdown_connections 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f58cc0b2990 0x7f58cc0b4ea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58e8109110 0x7f58e81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58e81025a0 0x7f58e819cee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 --2- 172.21.15.50:0/1459568181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58e8102ef0 0x7f58e819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.393+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 >> 172.21.15.50:0/1459568181 conn(0x7f58e80783b0 msgr2=0x7f58e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.394+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 shutdown_connections 2024-05-10T21:06:23.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:20.394+0000 7f58e7200700 1 -- 172.21.15.50:0/1459568181 wait complete. 2024-05-10T21:06:23.864 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:23 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/129318876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T21:06:23.865 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:23 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3376687672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T21:06:23.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:23 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/129318876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T21:06:23.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:23 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3376687672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T21:06:23.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:23 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/129318876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T21:06:23.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:23 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3376687672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T21:06:24.282 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645351 got 309237645355 for osd.11 2024-05-10T21:06:24.282 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:24.838 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:24 smithi050 ceph-mon[29546]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:24.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:24 smithi092 ceph-mon[34415]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:24.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:24 smithi142 ceph-mon[34846]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:25.787 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626496 got 180388626501 for osd.6 2024-05-10T21:06:25.787 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:26.880 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841578 got 283467841582 for osd.10 2024-05-10T21:06:26.880 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:26.970 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:26 smithi050 ceph-mon[29546]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:27.121 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:26 smithi092 ceph-mon[34415]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:27.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:26 smithi142 ceph-mon[34846]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:27.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:27 smithi050 ceph-mon[29546]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T21:06:27.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:27 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2153350048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T21:06:28.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:27 smithi092 ceph-mon[34415]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T21:06:28.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:27 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2153350048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T21:06:28.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:27 smithi142 ceph-mon[34846]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T21:06:28.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:27 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2153350048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T21:06:28.968 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:28 smithi050 ceph-mon[29546]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:06:29.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:28 smithi092 ceph-mon[34415]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:06:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:28 smithi142 ceph-mon[34846]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:06:30.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037809 got 257698037812 for osd.9 2024-05-10T21:06:30.423 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:30.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234039 got 231928234043 for osd.8 2024-05-10T21:06:30.530 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 -- 172.21.15.50:0/986759706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb80105460 msgr2=0x7fdb80105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 --2- 172.21.15.50:0/986759706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb80105460 0x7fdb80105870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009990 tx=0x7fdb70009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 -- 172.21.15.50:0/986759706 shutdown_connections 2024-05-10T21:06:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 --2- 172.21.15.50:0/986759706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb8010e8d0 0x7fdb80110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 --2- 172.21.15.50:0/986759706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb80105460 0x7fdb80105870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 --2- 172.21.15.50:0/986759706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105db0 0x7fdb8010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 -- 172.21.15.50:0/986759706 >> 172.21.15.50:0/986759706 conn(0x7fdb800fedc0 msgr2=0x7fdb801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 -- 172.21.15.50:0/986759706 shutdown_connections 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.400+0000 7fdb87c00700 1 -- 172.21.15.50:0/986759706 wait complete. 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.402+0000 7fdb87c00700 1 Processor -- start 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.402+0000 7fdb87c00700 1 -- start start 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.402+0000 7fdb87c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.402+0000 7fdb87c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 0x7fdb80103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.402+0000 7fdb87c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8010e8d0 0x7fdb80103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb87c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80114420 con 0x7fdb8010e8d0 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb87c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801142a0 con 0x7fdb80105460 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb87c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801145a0 con 0x7fdb80105db0 2024-05-10T21:06:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb85200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb85c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8010e8d0 0x7fdb80103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb85200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44040/0 (socket says 172.21.15.50:44040) 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb85200700 1 -- 172.21.15.50:0/2153350048 learned_addr learned my addr 172.21.15.50:0/2153350048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 0x7fdb80103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 -- 172.21.15.50:0/2153350048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 msgr2=0x7fdb80103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 -- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8010e8d0 msgr2=0x7fdb80103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8010e8d0 0x7fdb80103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb77e00700 1 -- 172.21.15.50:0/2153350048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801a9150 con 0x7fdb80105db0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.403+0000 7fdb85200700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb77e00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 0x7fdb80103690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9f0 tx=0x7fdb6c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00c9c0 con 0x7fdb80105db0 2024-05-10T21:06:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c00cb60 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801a9440 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c018680 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.404+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a9980 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.405+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb800512c0 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.408+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdb6c010430 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.409+0000 7fdb7e000700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.410+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fdb6c0e2710 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.410+0000 7fdb85200700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.410+0000 7fdb85200700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 0x7fdb640b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b2e0 tx=0x7fdb70004710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.411+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb6c0aa400 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.660+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdb8002e5f0 con 0x7fdb80105db0 2024-05-10T21:06:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.661+0000 7fdb7e000700 1 -- 172.21.15.50:0/2153350048 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb6c0a9d00 con 0x7fdb80105db0 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stdout:107374182487 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 msgr2=0x7fdb640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 0x7fdb640b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b2e0 tx=0x7fdb70004710 comp rx=0 tx=0).stop 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 msgr2=0x7fdb80103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 0x7fdb80103690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9f0 tx=0x7fdb6c00ed00 comp rx=0 tx=0).stop 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 shutdown_connections 2024-05-10T21:06:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdb640b2880 0x7fdb640b4d90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb80105460 0x7fdb80103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8010e8d0 0x7fdb80103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 --2- 172.21.15.50:0/2153350048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb80105db0 0x7fdb80103690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 >> 172.21.15.50:0/2153350048 conn(0x7fdb800fedc0 msgr2=0x7fdb800ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 shutdown_connections 2024-05-10T21:06:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:27.664+0000 7fdb87c00700 1 -- 172.21.15.50:0/2153350048 wait complete. 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 -- 172.21.15.50:0/1188028050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c10af50 msgr2=0x7ffb0c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 --2- 172.21.15.50:0/1188028050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c10af50 0x7ffb0c10b360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80099c0 tx=0x7ffaf8009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 -- 172.21.15.50:0/1188028050 shutdown_connections 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 --2- 172.21.15.50:0/1188028050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 0x7ffb0c113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 --2- 172.21.15.50:0/1188028050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c106620 0x7ffb0c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 --2- 172.21.15.50:0/1188028050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c10af50 0x7ffb0c10b360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.831+0000 7ffb10a00700 1 -- 172.21.15.50:0/1188028050 >> 172.21.15.50:0/1188028050 conn(0x7ffb0c102030 msgr2=0x7ffb0c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.832+0000 7ffb10a00700 1 -- 172.21.15.50:0/1188028050 shutdown_connections 2024-05-10T21:06:30.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.832+0000 7ffb10a00700 1 -- 172.21.15.50:0/1188028050 wait complete. 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.833+0000 7ffb10a00700 1 Processor -- start 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.833+0000 7ffb10a00700 1 -- start start 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 0x7ffb0c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 0x7ffb0c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 0x7ffb0c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c116520 con 0x7ffb0c10af50 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1163a0 con 0x7ffb0c10b8a0 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb10a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1166a0 con 0x7ffb0c106620 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb0aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 0x7ffb0c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 0x7ffb0c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb0aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 0x7ffb0c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36972/0 (socket says 172.21.15.50:36972) 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb0aa00700 1 -- 172.21.15.50:0/2116938702 learned_addr learned my addr 172.21.15.50:0/2116938702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.834+0000 7ffb0be00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 0x7ffb0c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 -- 172.21.15.50:0/2116938702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 msgr2=0x7ffb0c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 0x7ffb0c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 -- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 msgr2=0x7ffb0c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 0x7ffb0c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 -- 172.21.15.50:0/2116938702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c1a1d80 con 0x7ffb0c106620 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb0b400700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 0x7ffb0c19ceb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e9f0 tx=0x7ffafc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00c9c0 con 0x7ffb0c106620 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffafc00cb60 con 0x7ffb0c106620 2024-05-10T21:06:30.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c1a2070 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.835+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc018680 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.836+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c1a9990 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.837+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0c0512c0 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.839+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffafc010430 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.841+0000 7ffb03e00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 0x7ffaf00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.841+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffafc0e23b0 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.841+0000 7ffb0aa00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 0x7ffaf00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.842+0000 7ffb0aa00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 0x7ffaf00b4d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003760 tx=0x7ffaf801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.845+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffafc0aa0a0 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.092+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffb0c02e5f0 con 0x7ffb0c106620 2024-05-10T21:06:30.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.092+0000 7ffb03e00700 1 -- 172.21.15.50:0/2116938702 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffafc0b2120 con 0x7ffb0c106620 2024-05-10T21:06:30.706 INFO:teuthology.orchestra.run.smithi050.stdout:55834574949 2024-05-10T21:06:30.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.095+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 msgr2=0x7ffaf00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.095+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 0x7ffaf00b4d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8003760 tx=0x7ffaf801a040 comp rx=0 tx=0).stop 2024-05-10T21:06:30.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.095+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 msgr2=0x7ffb0c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.095+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 0x7ffb0c19ceb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e9f0 tx=0x7ffafc00ed00 comp rx=0 tx=0).stop 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 shutdown_connections 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffaf00b2880 0x7ffaf00b4d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb0c10b8a0 0x7ffb0c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb0c106620 0x7ffb0c19ceb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 --2- 172.21.15.50:0/2116938702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c10af50 0x7ffb0c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 >> 172.21.15.50:0/2116938702 conn(0x7ffb0c102030 msgr2=0x7ffb0c10a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 shutdown_connections 2024-05-10T21:06:30.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.096+0000 7ffb10a00700 1 -- 172.21.15.50:0/2116938702 wait complete. 2024-05-10T21:06:30.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:30 smithi050 ceph-mon[29546]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:30.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:30 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2116938702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T21:06:30.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:30 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1855274713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T21:06:31.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 -- 172.21.15.50:0/2568558172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f40764b0 msgr2=0x7fc9f4114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 --2- 172.21.15.50:0/2568558172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f40764b0 0x7fc9f4114e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009990 tx=0x7fc9e0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 -- 172.21.15.50:0/2568558172 shutdown_connections 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 --2- 172.21.15.50:0/2568558172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4115370 0x7fc9f4117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 --2- 172.21.15.50:0/2568558172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f4075b60 0x7fc9f4075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 --2- 172.21.15.50:0/2568558172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f40764b0 0x7fc9f4114e30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 -- 172.21.15.50:0/2568558172 >> 172.21.15.50:0/2568558172 conn(0x7fc9f4100040 msgr2=0x7fc9f4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 -- 172.21.15.50:0/2568558172 shutdown_connections 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.935+0000 7fc9fa200700 1 -- 172.21.15.50:0/2568558172 wait complete. 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.936+0000 7fc9fa200700 1 Processor -- start 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.936+0000 7fc9fa200700 1 -- start start 2024-05-10T21:06:31.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 0x7fc9f41a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 0x7fc9f41a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f411aa90 con 0x7fc9f4115370 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f411a910 con 0x7fc9f4075b60 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9fa200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f411ac10 con 0x7fc9f40764b0 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9f3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9f3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 0x7fc9f41a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9f3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36996/0 (socket says 172.21.15.50:36996) 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9f3e00700 1 -- 172.21.15.50:0/1855274713 learned_addr learned my addr 172.21.15.50:0/1855274713 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.937+0000 7fc9f2a00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 0x7fc9f41a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 -- 172.21.15.50:0/1855274713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 msgr2=0x7fc9f41a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 0x7fc9f41a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 -- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 msgr2=0x7fc9f41a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:31.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 -- 172.21.15.50:0/1855274713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f41a6330 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f2a00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 0x7fc9f41a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3e00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9f3400700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 0x7fc9f41a1460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400e9f0 tx=0x7fc9e400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4004220 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e40043c0 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f41a6620 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4005020 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.938+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f41adf40 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.940+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f40772a0 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.942+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc9e4004750 con 0x7fc9f4075b60 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.943+0000 7fc9ebe00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 0x7fc9d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:31.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.944+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc9e40e21b0 con 0x7fc9f4075b60 2024-05-10T21:06:31.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.944+0000 7fc9f2a00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 0x7fc9d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:31.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.945+0000 7fc9f2a00700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 0x7fc9d80b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098e0 tx=0x7fc9e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:133143986261 2024-05-10T21:06:31.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:29.948+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9e40a9ea0 con 0x7fc9f4075b60 2024-05-10T21:06:31.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.196+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc9f41a27c0 con 0x7fc9f4075b60 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.197+0000 7fc9ebe00700 1 -- 172.21.15.50:0/1855274713 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc9e40a97a0 con 0x7fc9f4075b60 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 msgr2=0x7fc9d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 0x7fc9d80b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098e0 tx=0x7fc9e0005960 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 msgr2=0x7fc9f41a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 0x7fc9f41a1460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400e9f0 tx=0x7fc9e400ed00 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 shutdown_connections 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9d80b2880 0x7fc9d80b4d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f4075b60 0x7fc9f41a1460 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f4115370 0x7fc9f41a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 --2- 172.21.15.50:0/1855274713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9f40764b0 0x7fc9f41a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.200+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 >> 172.21.15.50:0/1855274713 conn(0x7fc9f4100040 msgr2=0x7fc9f4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.201+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 shutdown_connections 2024-05-10T21:06:31.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:30.201+0000 7fc9fa200700 1 -- 172.21.15.50:0/1855274713 wait complete. 2024-05-10T21:06:31.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:30 smithi092 ceph-mon[34415]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:31.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:30 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2116938702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T21:06:31.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:30 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1855274713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T21:06:31.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:30 smithi142 ceph-mon[34846]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:31.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:30 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2116938702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T21:06:31.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:30 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1855274713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T21:06:31.757 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:31.770 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:32.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986254 got 133143986261 for osd.4 2024-05-10T21:06:32.263 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:32.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182482 got 107374182487 for osd.3 2024-05-10T21:06:32.939 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:33.009 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:32 smithi050 ceph-mon[29546]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:33.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:32 smithi092 ceph-mon[34415]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 -- 172.21.15.50:0/3413213916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab4106620 msgr2=0x7f2ab4106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 --2- 172.21.15.50:0/3413213916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab4106620 0x7f2ab4106a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80099c0 tx=0x7f2aa8009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 -- 172.21.15.50:0/3413213916 shutdown_connections 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 --2- 172.21.15.50:0/3413213916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410b8a0 0x7f2ab4113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 --2- 172.21.15.50:0/3413213916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab4106620 0x7f2ab4106a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 --2- 172.21.15.50:0/3413213916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab410af50 0x7f2ab410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 -- 172.21.15.50:0/3413213916 >> 172.21.15.50:0/3413213916 conn(0x7f2ab4102030 msgr2=0x7f2ab4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 -- 172.21.15.50:0/3413213916 shutdown_connections 2024-05-10T21:06:33.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.121+0000 7f2aba600700 1 -- 172.21.15.50:0/3413213916 wait complete. 2024-05-10T21:06:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.122+0000 7f2aba600700 1 Processor -- start 2024-05-10T21:06:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 -- start start 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab4106620 0x7f2ab419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410af50 0x7f2ab419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41164d0 con 0x7f2ab4106620 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4116350 con 0x7f2ab410af50 2024-05-10T21:06:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.123+0000 7f2aba600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4116650 con 0x7f2ab410b8a0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab4106620 0x7f2ab419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48982/0 (socket says 172.21.15.50:48982) 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 -- 172.21.15.50:0/270132191 learned_addr learned my addr 172.21.15.50:0/270132191 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab3e00700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410af50 0x7f2ab419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 -- 172.21.15.50:0/270132191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410af50 msgr2=0x7f2ab419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410af50 0x7f2ab419d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 -- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab4106620 msgr2=0x7f2ab419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab4106620 0x7f2ab419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.124+0000 7f2ab9c00700 1 -- 172.21.15.50:0/270132191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab41a1cf0 con 0x7f2ab410b8a0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2ab9c00700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00ef30 tx=0x7f2aac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:33.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0092e0 con 0x7f2ab410b8a0 2024-05-10T21:06:33.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aac004bb0 con 0x7f2ab410b8a0 2024-05-10T21:06:33.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004020 con 0x7f2ab410b8a0 2024-05-10T21:06:33.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41a1f80 con 0x7f2ab410b8a0 2024-05-10T21:06:33.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.125+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41a9900 con 0x7f2ab410b8a0 2024-05-10T21:06:33.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.126+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab40512c0 con 0x7f2ab410b8a0 2024-05-10T21:06:33.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.129+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2aac01b030 con 0x7f2ab410b8a0 2024-05-10T21:06:33.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:32 smithi142 ceph-mon[34846]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:33.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.130+0000 7f2ab1600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 0x7f2a9c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.131+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f2aac0e2620 con 0x7f2ab410b8a0 2024-05-10T21:06:33.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.131+0000 7f2ab9200700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 0x7f2a9c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.131+0000 7f2ab9200700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 0x7f2a9c0b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8000c00 tx=0x7f2aa801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:33.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.133+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2aac022020 con 0x7f2ab410b8a0 2024-05-10T21:06:33.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.389+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2ab402e5f0 con 0x7f2ab410b8a0 2024-05-10T21:06:33.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.389+0000 7f2ab1600700 1 -- 172.21.15.50:0/270132191 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2aac010040 con 0x7f2ab410b8a0 2024-05-10T21:06:33.391 INFO:teuthology.orchestra.run.smithi050.stdout:77309411424 2024-05-10T21:06:33.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 msgr2=0x7f2a9c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 0x7f2a9c0b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8000c00 tx=0x7f2aa801a040 comp rx=0 tx=0).stop 2024-05-10T21:06:33.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 msgr2=0x7f2ab41a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00ef30 tx=0x7f2aac00c3a0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 shutdown_connections 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2a9c0b2900 0x7f2a9c0b4e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab410b8a0 0x7f2ab41a1650 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ab4106620 0x7f2ab419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 --2- 172.21.15.50:0/270132191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab410af50 0x7f2ab419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.392+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 >> 172.21.15.50:0/270132191 conn(0x7f2ab4102030 msgr2=0x7f2ab410a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.393+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 shutdown_connections 2024-05-10T21:06:33.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.393+0000 7f2aba600700 1 -- 172.21.15.50:0/270132191 wait complete. 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 -- 172.21.15.50:0/2521624055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5641045f0 msgr2=0x7fe564104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 --2- 172.21.15.50:0/2521624055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5641045f0 0x7fe564104a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5540099c0 tx=0x7fe554009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 -- 172.21.15.50:0/2521624055 shutdown_connections 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 --2- 172.21.15.50:0/2521624055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564109870 0x7fe5641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 --2- 172.21.15.50:0/2521624055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5641045f0 0x7fe564104a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 --2- 172.21.15.50:0/2521624055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe564108f20 0x7fe564109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 -- 172.21.15.50:0/2521624055 >> 172.21.15.50:0/2521624055 conn(0x7fe564100040 msgr2=0x7fe564102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 -- 172.21.15.50:0/2521624055 shutdown_connections 2024-05-10T21:06:33.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.401+0000 7fe56a400700 1 -- 172.21.15.50:0/2521624055 wait complete. 2024-05-10T21:06:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.402+0000 7fe56a400700 1 Processor -- start 2024-05-10T21:06:33.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 -- start start 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5641045f0 0x7fe564075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe564109870 0x7fe564076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641144f0 con 0x7fe564108f20 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe564114370 con 0x7fe5641045f0 2024-05-10T21:06:33.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe56a400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe564114670 con 0x7fe564109870 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe562a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe562a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37030/0 (socket says 172.21.15.50:37030) 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.403+0000 7fe562a00700 1 -- 172.21.15.50:0/2719155013 learned_addr learned my addr 172.21.15.50:0/2719155013 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe563400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5641045f0 0x7fe564075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe563e00700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe564109870 0x7fe564076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe562a00700 1 -- 172.21.15.50:0/2719155013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe564109870 msgr2=0x7fe564076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe562a00700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe564109870 0x7fe564076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe562a00700 1 -- 172.21.15.50:0/2719155013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5641045f0 msgr2=0x7fe564075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe562a00700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5641045f0 0x7fe564075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.404+0000 7fe562a00700 1 -- 172.21.15.50:0/2719155013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5641a9240 con 0x7fe564108f20 2024-05-10T21:06:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe562a00700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe55000e970 tx=0x7fe55000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550004a30 con 0x7fe564108f20 2024-05-10T21:06:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55000cc90 con 0x7fe564108f20 2024-05-10T21:06:33.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5641a9530 con 0x7fe564108f20 2024-05-10T21:06:33.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550005610 con 0x7fe564108f20 2024-05-10T21:06:33.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.405+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5641a9a70 con 0x7fe564108f20 2024-05-10T21:06:33.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.406+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5640512c0 con 0x7fe564108f20 2024-05-10T21:06:33.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.411+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe55000c7e0 con 0x7fe564108f20 2024-05-10T21:06:33.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.413+0000 7fe55be00700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 0x7fe5480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:33.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.413+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fe5500e2910 con 0x7fe564108f20 2024-05-10T21:06:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.413+0000 7fe563400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 0x7fe5480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.414+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5500aa600 con 0x7fe564108f20 2024-05-10T21:06:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.414+0000 7fe563400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 0x7fe5480b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe554000c00 tx=0x7fe55401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:33.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.668+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe56402e5f0 con 0x7fe564108f20 2024-05-10T21:06:33.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.668+0000 7fe55be00700 1 -- 172.21.15.50:0/2719155013 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5500cc070 con 0x7fe564108f20 2024-05-10T21:06:33.670 INFO:teuthology.orchestra.run.smithi050.stdout:154618822734 2024-05-10T21:06:33.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.672+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 msgr2=0x7fe5480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.672+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 0x7fe5480b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe554000c00 tx=0x7fe55401a040 comp rx=0 tx=0).stop 2024-05-10T21:06:33.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.672+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 msgr2=0x7fe564076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:33.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.672+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe55000e970 tx=0x7fe55000ec80 comp rx=0 tx=0).stop 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 shutdown_connections 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5480b2880 0x7fe5480b4d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5641045f0 0x7fe564075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe564109870 0x7fe564076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 --2- 172.21.15.50:0/2719155013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe564108f20 0x7fe564076000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:33.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 >> 172.21.15.50:0/2719155013 conn(0x7fe564100040 msgr2=0x7fe5641086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:33.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 shutdown_connections 2024-05-10T21:06:33.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:33.673+0000 7fe56a400700 1 -- 172.21.15.50:0/2719155013 wait complete. 2024-05-10T21:06:33.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:33 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/270132191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T21:06:33.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:33 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2719155013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T21:06:34.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:33 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/270132191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T21:06:34.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:33 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2719155013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T21:06:34.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:33 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/270132191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T21:06:34.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:33 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2719155013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T21:06:34.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822729 got 154618822734 for osd.5 2024-05-10T21:06:34.598 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:35.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:34 smithi092 ceph-mon[34415]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:35.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:34 smithi142 ceph-mon[34846]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:35.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:34 smithi050 ceph-mon[29546]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:36.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574944 got 55834574949 for osd.1 2024-05-10T21:06:36.249 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:37.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:36 smithi050 ceph-mon[29546]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:37.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:36 smithi092 ceph-mon[34415]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:37.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:36 smithi142 ceph-mon[34846]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:06:37.280 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411424 for osd.2 2024-05-10T21:06:37.280 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:38.938 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:38 smithi050 ceph-mon[29546]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:39.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:38 smithi092 ceph-mon[34415]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:39.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:38 smithi142 ceph-mon[34846]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:06:39.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 -- 172.21.15.50:0/1924292704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 msgr2=0x7fcb381111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 --2- 172.21.15.50:0/1924292704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb381111d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b0d0 tx=0x7fcb3000b3e0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 -- 172.21.15.50:0/1924292704 shutdown_connections 2024-05-10T21:06:39.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 --2- 172.21.15.50:0/1924292704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb381045f0 0x7fcb38104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 --2- 172.21.15.50:0/1924292704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb381111d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 --2- 172.21.15.50:0/1924292704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb38108f20 0x7fcb38109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 -- 172.21.15.50:0/1924292704 >> 172.21.15.50:0/1924292704 conn(0x7fcb380ffec0 msgr2=0x7fcb38102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 -- 172.21.15.50:0/1924292704 shutdown_connections 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.675+0000 7fcb3da00700 1 -- 172.21.15.50:0/1924292704 wait complete. 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.676+0000 7fcb3da00700 1 Processor -- start 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.676+0000 7fcb3da00700 1 -- start start 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb38108f20 0x7fcb3819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb3819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381144f0 con 0x7fcb381045f0 2024-05-10T21:06:39.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38114370 con 0x7fcb38108f20 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb3da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38114670 con 0x7fcb38109870 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb36a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb36a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33518/0 (socket says 172.21.15.50:33518) 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb36a00700 1 -- 172.21.15.50:0/47316510 learned_addr learned my addr 172.21.15.50:0/47316510 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb2fe00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb38108f20 0x7fcb3819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.677+0000 7fcb37400700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb3819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 -- 172.21.15.50:0/47316510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 msgr2=0x7fcb3819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb3819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 -- 172.21.15.50:0/47316510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb38108f20 msgr2=0x7fcb3819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb38108f20 0x7fcb3819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 -- 172.21.15.50:0/47316510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3819fd30 con 0x7fcb381045f0 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb37400700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb3819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:06:39.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb36a00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e9a0 tx=0x7fcb2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800cbd0 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2800cd70 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381a0020 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28018680 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.678+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb381a7940 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.680+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb380512c0 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.684+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcb28010430 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.685+0000 7fcb2f400700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 0x7fcb1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.686+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fcb280e2970 con 0x7fcb381045f0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.686+0000 7fcb2fe00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 0x7fcb1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.686+0000 7fcb2fe00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 0x7fcb1c0b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb381034d0 tx=0x7fcb20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:39.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.689+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb280aa660 con 0x7fcb381045f0 2024-05-10T21:06:39.385 INFO:teuthology.orchestra.run.smithi050.stdout:34359738475 2024-05-10T21:06:39.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.936+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcb3802e5f0 con 0x7fcb381045f0 2024-05-10T21:06:39.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.937+0000 7fcb2f400700 1 -- 172.21.15.50:0/47316510 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb280a9f60 con 0x7fcb381045f0 2024-05-10T21:06:39.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 msgr2=0x7fcb1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 0x7fcb1c0b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb381034d0 tx=0x7fcb20009260 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 msgr2=0x7fcb3819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e9a0 tx=0x7fcb2800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 shutdown_connections 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcb1c0b2880 0x7fcb1c0b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb38109870 0x7fcb3819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb381045f0 0x7fcb3819ae60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 --2- 172.21.15.50:0/47316510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb38108f20 0x7fcb3819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 >> 172.21.15.50:0/47316510 conn(0x7fcb380ffec0 msgr2=0x7fcb38108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.941+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 shutdown_connections 2024-05-10T21:06:39.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:38.942+0000 7fcb3da00700 1 -- 172.21.15.50:0/47316510 wait complete. 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 -- 172.21.15.50:0/4012646292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 msgr2=0x7fbb641038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 --2- 172.21.15.50:0/4012646292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb641038d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099c0 tx=0x7fbb58009cd0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 -- 172.21.15.50:0/4012646292 shutdown_connections 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 --2- 172.21.15.50:0/4012646292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64108740 0x7fbb641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 --2- 172.21.15.50:0/4012646292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb641038d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 --2- 172.21.15.50:0/4012646292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64107df0 0x7fbb64108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 -- 172.21.15.50:0/4012646292 >> 172.21.15.50:0/4012646292 conn(0x7fbb640fedc0 msgr2=0x7fbb641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:39.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 -- 172.21.15.50:0/4012646292 shutdown_connections 2024-05-10T21:06:39.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.399+0000 7fbb6c000700 1 -- 172.21.15.50:0/4012646292 wait complete. 2024-05-10T21:06:39.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 Processor -- start 2024-05-10T21:06:39.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 -- start start 2024-05-10T21:06:39.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 0x7fbb64199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64108740 0x7fbb6419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64113370 con 0x7fbb641034c0 2024-05-10T21:06:39.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641131f0 con 0x7fbb64108740 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.401+0000 7fbb6c000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641134f0 con 0x7fbb64107df0 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb69600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb69600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33530/0 (socket says 172.21.15.50:33530) 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb69600700 1 -- 172.21.15.50:0/497529500 learned_addr learned my addr 172.21.15.50:0/497529500 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb6a000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64108740 0x7fbb6419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 0x7fbb64199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 -- 172.21.15.50:0/497529500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64108740 msgr2=0x7fbb6419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64108740 0x7fbb6419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 -- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 msgr2=0x7fbb64199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb68c00700 1 -- 172.21.15.50:0/497529500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641a9180 con 0x7fbb64107df0 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.402+0000 7fbb69600700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb68c00700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 0x7fbb64199fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e9c0 tx=0x7fbb5000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000c9c0 con 0x7fbb64107df0 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641a9470 con 0x7fbb64107df0 2024-05-10T21:06:39.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5000cb60 con 0x7fbb64107df0 2024-05-10T21:06:39.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50018680 con 0x7fbb64107df0 2024-05-10T21:06:39.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.403+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641a9a00 con 0x7fbb64107df0 2024-05-10T21:06:39.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.404+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb44005330 con 0x7fbb64107df0 2024-05-10T21:06:39.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.407+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbb50018820 con 0x7fbb64107df0 2024-05-10T21:06:39.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.409+0000 7fbb62000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 0x7fbb4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:39.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.409+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fbb500e2660 con 0x7fbb64107df0 2024-05-10T21:06:39.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.409+0000 7fbb69600700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 0x7fbb4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:39.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.410+0000 7fbb69600700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 0x7fbb4c0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009990 tx=0x7fbb5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:39.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.411+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb500aa350 con 0x7fbb64107df0 2024-05-10T21:06:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.657+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbb440058f0 con 0x7fbb64107df0 2024-05-10T21:06:39.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.657+0000 7fbb62000700 1 -- 172.21.15.50:0/497529500 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb500b23d0 con 0x7fbb64107df0 2024-05-10T21:06:39.662 INFO:teuthology.orchestra.run.smithi050.stdout:206158430277 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 msgr2=0x7fbb4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 0x7fbb4c0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009990 tx=0x7fbb5801a040 comp rx=0 tx=0).stop 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 msgr2=0x7fbb64199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 0x7fbb64199fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e9c0 tx=0x7fbb5000ecd0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 shutdown_connections 2024-05-10T21:06:39.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbb4c0b2880 0x7fbb4c0b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb64108740 0x7fbb6419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb641034c0 0x7fbb64199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 --2- 172.21.15.50:0/497529500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64107df0 0x7fbb64199fc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 >> 172.21.15.50:0/497529500 conn(0x7fbb640fedc0 msgr2=0x7fbb641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.661+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 shutdown_connections 2024-05-10T21:06:39.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:39.662+0000 7fbb6c000700 1 -- 172.21.15.50:0/497529500 wait complete. 2024-05-10T21:06:39.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:39 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/47316510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T21:06:39.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:39 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/497529500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T21:06:40.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:39 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/47316510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T21:06:40.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:39 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/497529500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T21:06:40.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:39 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/47316510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T21:06:40.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:39 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/497529500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T21:06:40.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738470 got 34359738475 for osd.0 2024-05-10T21:06:40.432 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:40.937 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:40 smithi050 ceph-mon[29546]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:06:40.941 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430271 got 206158430277 for osd.7 2024-05-10T21:06:40.942 DEBUG:teuthology.parallel:result is None 2024-05-10T21:06:40.942 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T21:06:40.942 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:06:41.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:40 smithi092 ceph-mon[34415]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:06:41.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:40 smithi142 ceph-mon[34846]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:06:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:42.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:42 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:42 smithi050 ceph-mon[29546]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:42.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:42 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.871 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:42 smithi092 ceph-mon[34415]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:42.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:42 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:42.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:42 smithi142 ceph-mon[34846]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.660+0000 7f6a87e00700 1 -- 172.21.15.50:0/2075323002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80103a50 msgr2=0x7f6a80110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.660+0000 7f6a87e00700 1 --2- 172.21.15.50:0/2075323002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80103a50 0x7f6a80110770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009990 tx=0x7f6a6c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 -- 172.21.15.50:0/2075323002 shutdown_connections 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 --2- 172.21.15.50:0/2075323002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80110cb0 0x7f6a801130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 --2- 172.21.15.50:0/2075323002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103100 0x7f6a80103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 --2- 172.21.15.50:0/2075323002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80103a50 0x7f6a80110770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 -- 172.21.15.50:0/2075323002 >> 172.21.15.50:0/2075323002 conn(0x7f6a800fedf0 msgr2=0x7f6a80101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 -- 172.21.15.50:0/2075323002 shutdown_connections 2024-05-10T21:06:43.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.661+0000 7f6a87e00700 1 -- 172.21.15.50:0/2075323002 wait complete. 2024-05-10T21:06:43.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:06:43.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:06:43.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.662+0000 7f6a87e00700 1 Processor -- start 2024-05-10T21:06:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 -- start start 2024-05-10T21:06:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80103100 0x7f6a8019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:43.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 0x7f6a801a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80116660 con 0x7f6a80110cb0 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801164e0 con 0x7f6a80103100 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.663+0000 7f6a87e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801167e0 con 0x7f6a80103a50 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a85e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 0x7f6a801a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a85400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80103100 0x7f6a8019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41054/0 (socket says 172.21.15.50:41054) 2024-05-10T21:06:43.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 -- 172.21.15.50:0/398996810 learned_addr learned my addr 172.21.15.50:0/398996810 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 -- 172.21.15.50:0/398996810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80103100 msgr2=0x7f6a8019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80103100 0x7f6a8019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 -- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 msgr2=0x7f6a801a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 0x7f6a801a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a84a00700 1 -- 172.21.15.50:0/398996810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a801a1ee0 con 0x7f6a80103a50 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.664+0000 7f6a85e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 0x7f6a801a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:43.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a84a00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c000c00 tx=0x7f6a6c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c026070 con 0x7f6a80103a50 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a801a2170 con 0x7f6a80103a50 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a6c021410 con 0x7f6a80103a50 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c017650 con 0x7f6a80103a50 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.665+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801a9af0 con 0x7f6a80103a50 2024-05-10T21:06:43.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.666+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a80076250 con 0x7f6a80103a50 2024-05-10T21:06:43.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.669+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a6c029070 con 0x7f6a80103a50 2024-05-10T21:06:43.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.671+0000 7f6a7e000700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 0x7f6a680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:43.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.671+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6a6c0e2f00 con 0x7f6a80103a50 2024-05-10T21:06:43.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.671+0000 7f6a85400700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 0x7f6a680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:43.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.672+0000 7f6a85400700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 0x7f6a680b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009c60 tx=0x7f6a74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:43.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.675+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a6c013070 con 0x7f6a80103a50 2024-05-10T21:06:43.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:06:43.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:06:43.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:43.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:06:43.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:06:43.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:06:43.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.912+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a8019e370 con 0x7f6a680b2900 2024-05-10T21:06:43.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.927+0000 7f6a7e000700 1 -- 172.21.15.50:0/398996810 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258817 (secure 0 0 0) 0x7f6a8019e370 con 0x7f6a680b2900 2024-05-10T21:06:43.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.934+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 msgr2=0x7f6a680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.934+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 0x7f6a680b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009c60 tx=0x7f6a74009260 comp rx=0 tx=0).stop 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.934+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 msgr2=0x7f6a8019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.934+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c000c00 tx=0x7f6a6c009510 comp rx=0 tx=0).stop 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 shutdown_connections 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a680b2900 0x7f6a680b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a80103100 0x7f6a8019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a80110cb0 0x7f6a801a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 --2- 172.21.15.50:0/398996810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80103a50 0x7f6a8019d5b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:43.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 >> 172.21.15.50:0/398996810 conn(0x7f6a800fedf0 msgr2=0x7f6a80112770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:43.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 shutdown_connections 2024-05-10T21:06:43.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:43.935+0000 7f6a87e00700 1 -- 172.21.15.50:0/398996810 wait complete. 2024-05-10T21:06:43.939 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:06:44.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:44 smithi050 ceph-mon[29546]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:44.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:44 smithi050 ceph-mon[29546]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:44.831 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-05-10T21:06:43.624561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659203,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":795,"num_read_kb":498,"num_write":299,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374184,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567640,"statfs":{"total":1151940427776,"available":1151557263360,"internally_reserved":0,"allocated":12853248,"data_stored":7214925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004801"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578816+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:25.578816+0000","last_peered":"2024-05-10T21:06:25.578816+0000","last_clean":"2024-05-10T21:06:25.578816+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:25.578816+0000","last_undegraded":"2024-05-10T21:06:25.578816+0000","last_fullsized":"2024-05-10T21:06:25.578816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580291+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:25.580291+0000","last_peered":"2024-05-10T21:06:25.580291+0000","last_clean":"2024-05-10T21:06:25.580291+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:25.580291+0000","last_undegraded":"2024-05-10T21:06:25.580291+0000","last_fullsized":"2024-05-10T21:06:25.580291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581398+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:25.581398+0000","last_peered":"2024-05-10T21:06:25.581398+0000","last_clean":"2024-05-10T21:06:25.581398+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:25.581398+0000","last_undegraded":"2024-05-10T21:06:25.581398+0000","last_fullsized":"2024-05-10T21:06:25.581398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556625+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556625+0000","last_peered":"2024-05-10T21:06:25.556625+0000","last_clean":"2024-05-10T21:06:25.556625+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556625+0000","last_undegraded":"2024-05-10T21:06:25.556625+0000","last_fullsized":"2024-05-10T21:06:25.556625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.563869+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.563869+0000","last_peered":"2024-05-10T21:06:25.563869+0000","last_clean":"2024-05-10T21:06:25.563869+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.563869+0000","last_undegraded":"2024-05-10T21:06:25.563869+0000","last_fullsized":"2024-05-10T21:06:25.563869+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562418+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562418+0000","last_peered":"2024-05-10T21:06:25.562418+0000","last_clean":"2024-05-10T21:06:25.562418+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562418+0000","last_undegraded":"2024-05-10T21:06:25.562418+0000","last_fullsized":"2024-05-10T21:06:25.562418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557802+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.557802+0000","last_peered":"2024-05-10T21:06:25.557802+0000","last_clean":"2024-05-10T21:06:25.557802+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.557802+0000","last_undegraded":"2024-05-10T21:06:25.557802+0000","last_fullsized":"2024-05-10T21:06:25.557802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559486+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.559486+0000","last_peered":"2024-05-10T21:06:25.559486+0000","last_clean":"2024-05-10T21:06:25.559486+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.559486+0000","last_undegraded":"2024-05-10T21:06:25.559486+0000","last_fullsized":"2024-05-10T21:06:25.559486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567789+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567789+0000","last_peered":"2024-05-10T21:06:25.567789+0000","last_clean":"2024-05-10T21:06:25.567789+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567789+0000","last_undegraded":"2024-05-10T21:06:25.567789+0000","last_fullsized":"2024-05-10T21:06:25.567789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566659+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566659+0000","last_peered":"2024-05-10T21:06:25.566659+0000","last_clean":"2024-05-10T21:06:25.566659+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566659+0000","last_undegraded":"2024-05-10T21:06:25.566659+0000","last_fullsized":"2024-05-10T21:06:25.566659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565929+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.565929+0000","last_peered":"2024-05-10T21:06:25.565929+0000","last_clean":"2024-05-10T21:06:25.565929+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.565929+0000","last_undegraded":"2024-05-10T21:06:25.565929+0000","last_fullsized":"2024-05-10T21:06:25.565929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558019+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558019+0000","last_peered":"2024-05-10T21:06:25.558019+0000","last_clean":"2024-05-10T21:06:25.558019+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558019+0000","last_undegraded":"2024-05-10T21:06:25.558019+0000","last_fullsized":"2024-05-10T21:06:25.558019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576251+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576251+0000","last_peered":"2024-05-10T21:06:25.576251+0000","last_clean":"2024-05-10T21:06:25.576251+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576251+0000","last_undegraded":"2024-05-10T21:06:25.576251+0000","last_fullsized":"2024-05-10T21:06:25.576251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578436+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.578436+0000","last_peered":"2024-05-10T21:06:25.578436+0000","last_clean":"2024-05-10T21:06:25.578436+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.578436+0000","last_undegraded":"2024-05-10T21:06:25.578436+0000","last_fullsized":"2024-05-10T21:06:25.578436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579431+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579431+0000","last_peered":"2024-05-10T21:06:25.579431+0000","last_clean":"2024-05-10T21:06:25.579431+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579431+0000","last_undegraded":"2024-05-10T21:06:25.579431+0000","last_fullsized":"2024-05-10T21:06:25.579431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581089+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581089+0000","last_peered":"2024-05-10T21:06:25.581089+0000","last_clean":"2024-05-10T21:06:25.581089+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581089+0000","last_undegraded":"2024-05-10T21:06:25.581089+0000","last_fullsized":"2024-05-10T21:06:25.581089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578810+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578810+0000","last_peered":"2024-05-10T21:06:25.578810+0000","last_clean":"2024-05-10T21:06:25.578810+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578810+0000","last_undegraded":"2024-05-10T21:06:25.578810+0000","last_fullsized":"2024-05-10T21:06:25.578810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567776+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567776+0000","last_peered":"2024-05-10T21:06:25.567776+0000","last_clean":"2024-05-10T21:06:25.567776+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567776+0000","last_undegraded":"2024-05-10T21:06:25.567776+0000","last_fullsized":"2024-05-10T21:06:25.567776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.184551+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.184551+0000","last_peered":"2024-05-10T21:04:57.184551+0000","last_clean":"2024-05-10T21:04:57.184551+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.184551+0000","last_undegraded":"2024-05-10T21:04:57.184551+0000","last_fullsized":"2024-05-10T21:04:57.184551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569551+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.569551+0000","last_peered":"2024-05-10T21:06:25.569551+0000","last_clean":"2024-05-10T21:06:25.569551+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.569551+0000","last_undegraded":"2024-05-10T21:06:25.569551+0000","last_fullsized":"2024-05-10T21:06:25.569551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549962+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:04:57.549962+0000","last_peered":"2024-05-10T21:04:57.549962+0000","last_clean":"2024-05-10T21:04:57.549962+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:04:57.549962+0000","last_undegraded":"2024-05-10T21:04:57.549962+0000","last_fullsized":"2024-05-10T21:04:57.549962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581364+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:25.581364+0000","last_peered":"2024-05-10T21:06:25.581364+0000","last_clean":"2024-05-10T21:06:25.581364+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:25.581364+0000","last_undegraded":"2024-05-10T21:06:25.581364+0000","last_fullsized":"2024-05-10T21:06:25.581364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891580+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891580+0000","last_peered":"2024-05-10T21:04:56.891580+0000","last_clean":"2024-05-10T21:04:56.891580+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891580+0000","last_undegraded":"2024-05-10T21:04:56.891580+0000","last_fullsized":"2024-05-10T21:04:56.891580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572211+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572211+0000","last_peered":"2024-05-10T21:06:25.572211+0000","last_clean":"2024-05-10T21:06:25.572211+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572211+0000","last_undegraded":"2024-05-10T21:06:25.572211+0000","last_fullsized":"2024-05-10T21:06:25.572211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891333+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891333+0000","last_peered":"2024-05-10T21:04:56.891333+0000","last_clean":"2024-05-10T21:04:56.891333+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891333+0000","last_undegraded":"2024-05-10T21:04:56.891333+0000","last_fullsized":"2024-05-10T21:04:56.891333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.563861+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.563861+0000","last_peered":"2024-05-10T21:06:25.563861+0000","last_clean":"2024-05-10T21:06:25.563861+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.563861+0000","last_undegraded":"2024-05-10T21:06:25.563861+0000","last_fullsized":"2024-05-10T21:06:25.563861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:38.943437+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:06:38.943437+0000","last_peered":"2024-05-10T21:06:38.943437+0000","last_clean":"2024-05-10T21:06:38.943437+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:06:38.943437+0000","last_undegraded":"2024-05-10T21:06:38.943437+0000","last_fullsized":"2024-05-10T21:06:38.943437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555871+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.555871+0000","last_peered":"2024-05-10T21:06:25.555871+0000","last_clean":"2024-05-10T21:06:25.555871+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.555871+0000","last_undegraded":"2024-05-10T21:06:25.555871+0000","last_fullsized":"2024-05-10T21:06:25.555871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.885924+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.885924+0000","last_peered":"2024-05-10T21:04:56.885924+0000","last_clean":"2024-05-10T21:04:56.885924+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.885924+0000","last_undegraded":"2024-05-10T21:04:56.885924+0000","last_fullsized":"2024-05-10T21:04:56.885924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572513+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572513+0000","last_peered":"2024-05-10T21:06:25.572513+0000","last_clean":"2024-05-10T21:06:25.572513+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572513+0000","last_undegraded":"2024-05-10T21:06:25.572513+0000","last_fullsized":"2024-05-10T21:06:25.572513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170499+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170499+0000","last_peered":"2024-05-10T21:04:57.170499+0000","last_clean":"2024-05-10T21:04:57.170499+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170499+0000","last_undegraded":"2024-05-10T21:04:57.170499+0000","last_fullsized":"2024-05-10T21:04:57.170499+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574600+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574600+0000","last_peered":"2024-05-10T21:06:25.574600+0000","last_clean":"2024-05-10T21:06:25.574600+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574600+0000","last_undegraded":"2024-05-10T21:06:25.574600+0000","last_fullsized":"2024-05-10T21:06:25.574600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330788+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330788+0000","last_peered":"2024-05-10T21:04:57.330788+0000","last_clean":"2024-05-10T21:04:57.330788+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330788+0000","last_undegraded":"2024-05-10T21:04:57.330788+0000","last_fullsized":"2024-05-10T21:04:57.330788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575469+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575469+0000","last_peered":"2024-05-10T21:06:25.575469+0000","last_clean":"2024-05-10T21:06:25.575469+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575469+0000","last_undegraded":"2024-05-10T21:06:25.575469+0000","last_fullsized":"2024-05-10T21:06:25.575469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:40.945554+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:06:40.945554+0000","last_peered":"2024-05-10T21:06:40.945554+0000","last_clean":"2024-05-10T21:06:40.945554+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:06:40.945554+0000","last_undegraded":"2024-05-10T21:06:40.945554+0000","last_fullsized":"2024-05-10T21:06:40.945554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568310+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568310+0000","last_peered":"2024-05-10T21:06:25.568310+0000","last_clean":"2024-05-10T21:06:25.568310+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568310+0000","last_undegraded":"2024-05-10T21:06:25.568310+0000","last_fullsized":"2024-05-10T21:06:25.568310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896142+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896142+0000","last_peered":"2024-05-10T21:04:56.896142+0000","last_clean":"2024-05-10T21:04:56.896142+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896142+0000","last_undegraded":"2024-05-10T21:04:56.896142+0000","last_fullsized":"2024-05-10T21:04:56.896142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.580506+0000","last_peered":"2024-05-10T21:06:25.580506+0000","last_clean":"2024-05-10T21:06:25.580506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.580506+0000","last_undegraded":"2024-05-10T21:06:25.580506+0000","last_fullsized":"2024-05-10T21:06:25.580506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330748+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330748+0000","last_peered":"2024-05-10T21:04:57.330748+0000","last_clean":"2024-05-10T21:04:57.330748+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330748+0000","last_undegraded":"2024-05-10T21:04:57.330748+0000","last_fullsized":"2024-05-10T21:04:57.330748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564663+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.564663+0000","last_peered":"2024-05-10T21:06:25.564663+0000","last_clean":"2024-05-10T21:06:25.564663+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.564663+0000","last_undegraded":"2024-05-10T21:06:25.564663+0000","last_fullsized":"2024-05-10T21:06:25.564663+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":305,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:41.947442+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:06:41.947442+0000","last_peered":"2024-05-10T21:06:41.947442+0000","last_clean":"2024-05-10T21:06:41.947442+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:06:41.947442+0000","last_undegraded":"2024-05-10T21:06:41.947442+0000","last_fullsized":"2024-05-10T21:06:41.947442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":244,"num_read_kb":185,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552519+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.552519+0000","last_peered":"2024-05-10T21:06:25.552519+0000","last_clean":"2024-05-10T21:06:25.552519+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.552519+0000","last_undegraded":"2024-05-10T21:06:25.552519+0000","last_fullsized":"2024-05-10T21:06:25.552519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574330+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574330+0000","last_peered":"2024-05-10T21:06:25.574330+0000","last_clean":"2024-05-10T21:06:25.574330+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574330+0000","last_undegraded":"2024-05-10T21:06:25.574330+0000","last_fullsized":"2024-05-10T21:06:25.574330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576912+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.576912+0000","last_peered":"2024-05-10T21:06:25.576912+0000","last_clean":"2024-05-10T21:06:25.576912+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.576912+0000","last_undegraded":"2024-05-10T21:06:25.576912+0000","last_fullsized":"2024-05-10T21:06:25.576912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:36.996626+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:06:36.996626+0000","last_peered":"2024-05-10T21:06:36.996626+0000","last_clean":"2024-05-10T21:06:36.996626+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:06:36.996626+0000","last_undegraded":"2024-05-10T21:06:36.996626+0000","last_fullsized":"2024-05-10T21:06:36.996626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549413+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549413+0000","last_peered":"2024-05-10T21:06:25.549413+0000","last_clean":"2024-05-10T21:06:25.549413+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549413+0000","last_undegraded":"2024-05-10T21:06:25.549413+0000","last_fullsized":"2024-05-10T21:06:25.549413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553249+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.553249+0000","last_peered":"2024-05-10T21:06:25.553249+0000","last_clean":"2024-05-10T21:06:25.553249+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.553249+0000","last_undegraded":"2024-05-10T21:06:25.553249+0000","last_fullsized":"2024-05-10T21:06:25.553249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893043+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.893043+0000","last_peered":"2024-05-10T21:04:56.893043+0000","last_clean":"2024-05-10T21:04:56.893043+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.893043+0000","last_undegraded":"2024-05-10T21:04:56.893043+0000","last_fullsized":"2024-05-10T21:04:56.893043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576512+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:25.576512+0000","last_peered":"2024-05-10T21:06:25.576512+0000","last_clean":"2024-05-10T21:06:25.576512+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:25.576512+0000","last_undegraded":"2024-05-10T21:06:25.576512+0000","last_fullsized":"2024-05-10T21:06:25.576512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895673+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895673+0000","last_peered":"2024-05-10T21:04:56.895673+0000","last_clean":"2024-05-10T21:04:56.895673+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895673+0000","last_undegraded":"2024-05-10T21:04:56.895673+0000","last_fullsized":"2024-05-10T21:04:56.895673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578224+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:25.578224+0000","last_peered":"2024-05-10T21:06:25.578224+0000","last_clean":"2024-05-10T21:06:25.578224+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:25.578224+0000","last_undegraded":"2024-05-10T21:06:25.578224+0000","last_fullsized":"2024-05-10T21:06:25.578224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555147+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.555147+0000","last_peered":"2024-05-10T21:06:25.555147+0000","last_clean":"2024-05-10T21:06:25.555147+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.555147+0000","last_undegraded":"2024-05-10T21:06:25.555147+0000","last_fullsized":"2024-05-10T21:06:25.555147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:40.002202+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:06:40.002202+0000","last_peered":"2024-05-10T21:06:40.002202+0000","last_clean":"2024-05-10T21:06:40.002202+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:06:40.002202+0000","last_undegraded":"2024-05-10T21:06:40.002202+0000","last_fullsized":"2024-05-10T21:06:40.002202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570231+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:25.570231+0000","last_peered":"2024-05-10T21:06:25.570231+0000","last_clean":"2024-05-10T21:06:25.570231+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:25.570231+0000","last_undegraded":"2024-05-10T21:06:25.570231+0000","last_fullsized":"2024-05-10T21:06:25.570231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331885+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331885+0000","last_peered":"2024-05-10T21:04:57.331885+0000","last_clean":"2024-05-10T21:04:57.331885+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331885+0000","last_undegraded":"2024-05-10T21:04:57.331885+0000","last_fullsized":"2024-05-10T21:04:57.331885+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895557+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:04:56.895557+0000","last_peered":"2024-05-10T21:04:56.895557+0000","last_clean":"2024-05-10T21:04:56.895557+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:04:56.895557+0000","last_undegraded":"2024-05-10T21:04:56.895557+0000","last_fullsized":"2024-05-10T21:04:56.895557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552134+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:25.552134+0000","last_peered":"2024-05-10T21:06:25.552134+0000","last_clean":"2024-05-10T21:06:25.552134+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:25.552134+0000","last_undegraded":"2024-05-10T21:06:25.552134+0000","last_fullsized":"2024-05-10T21:06:25.552134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895689+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:04:56.895689+0000","last_peered":"2024-05-10T21:04:56.895689+0000","last_clean":"2024-05-10T21:04:56.895689+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:04:56.895689+0000","last_undegraded":"2024-05-10T21:04:56.895689+0000","last_fullsized":"2024-05-10T21:04:56.895689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891596+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891596+0000","last_peered":"2024-05-10T21:04:56.891596+0000","last_clean":"2024-05-10T21:04:56.891596+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891596+0000","last_undegraded":"2024-05-10T21:04:56.891596+0000","last_fullsized":"2024-05-10T21:04:56.891596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562727+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:25.562727+0000","last_peered":"2024-05-10T21:06:25.562727+0000","last_clean":"2024-05-10T21:06:25.562727+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:25.562727+0000","last_undegraded":"2024-05-10T21:06:25.562727+0000","last_fullsized":"2024-05-10T21:06:25.562727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.968140+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:25.968140+0000","last_peered":"2024-05-10T21:06:25.968140+0000","last_clean":"2024-05-10T21:06:25.968140+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:25.968140+0000","last_undegraded":"2024-05-10T21:06:25.968140+0000","last_fullsized":"2024-05-10T21:06:25.968140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.340707+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.340707+0000","last_peered":"2024-05-10T21:04:57.340707+0000","last_clean":"2024-05-10T21:04:57.340707+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.340707+0000","last_undegraded":"2024-05-10T21:04:57.340707+0000","last_fullsized":"2024-05-10T21:04:57.340707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557980+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:25.557980+0000","last_peered":"2024-05-10T21:06:25.557980+0000","last_clean":"2024-05-10T21:06:25.557980+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:25.557980+0000","last_undegraded":"2024-05-10T21:06:25.557980+0000","last_fullsized":"2024-05-10T21:06:25.557980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552796+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:25.552796+0000","last_peered":"2024-05-10T21:06:25.552796+0000","last_clean":"2024-05-10T21:06:25.552796+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:25.552796+0000","last_undegraded":"2024-05-10T21:06:25.552796+0000","last_fullsized":"2024-05-10T21:06:25.552796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569414+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:25.569414+0000","last_peered":"2024-05-10T21:06:25.569414+0000","last_clean":"2024-05-10T21:06:25.569414+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:25.569414+0000","last_undegraded":"2024-05-10T21:06:25.569414+0000","last_fullsized":"2024-05-10T21:06:25.569414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579761+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:25.579761+0000","last_peered":"2024-05-10T21:06:25.579761+0000","last_clean":"2024-05-10T21:06:25.579761+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:25.579761+0000","last_undegraded":"2024-05-10T21:06:25.579761+0000","last_fullsized":"2024-05-10T21:06:25.579761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68601,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":719,"num_read_kb":434,"num_write":174,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645360,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841587,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1093497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037816,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430278,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626506,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":501484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":502140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822736,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:06:44.836 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:06:44.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:44 smithi092 ceph-mon[34415]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:44.870 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:44 smithi092 ceph-mon[34415]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:44.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:44 smithi142 ceph-mon[34846]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:44.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:44 smithi142 ceph-mon[34846]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:45.353 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:47.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:46 smithi050 ceph-mon[29546]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:47.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:46 smithi092 ceph-mon[34415]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:47.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:46 smithi142 ceph-mon[34846]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.619+0000 7fc5b2c00700 1 -- 172.21.15.50:0/1768222688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac10af50 msgr2=0x7fc5ac10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.619+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/1768222688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac10af50 0x7fc5ac10b360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc598009920 tx=0x7fc598009c30 comp rx=0 tx=0).stop 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 -- 172.21.15.50:0/1768222688 shutdown_connections 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/1768222688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10b8a0 0x7fc5ac113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/1768222688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac106620 0x7fc5ac106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/1768222688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac10af50 0x7fc5ac10b360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 -- 172.21.15.50:0/1768222688 >> 172.21.15.50:0/1768222688 conn(0x7fc5ac102030 msgr2=0x7fc5ac104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 -- 172.21.15.50:0/1768222688 shutdown_connections 2024-05-10T21:06:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.620+0000 7fc5b2c00700 1 -- 172.21.15.50:0/1768222688 wait complete. 2024-05-10T21:06:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.622+0000 7fc5b2c00700 1 Processor -- start 2024-05-10T21:06:47.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.622+0000 7fc5b2c00700 1 -- start start 2024-05-10T21:06:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 0x7fc5ac19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 0x7fc5ac19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac10b8a0 0x7fc5ac1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1164b0 con 0x7fc5ac10b8a0 2024-05-10T21:06:47.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac116330 con 0x7fc5ac10af50 2024-05-10T21:06:47.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac116630 con 0x7fc5ac106620 2024-05-10T21:06:47.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac10b8a0 0x7fc5ac1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 0x7fc5ac19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 0x7fc5ac19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58280/0 (socket says 172.21.15.50:58280) 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b0e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 0x7fc5ac19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.623+0000 7fc5b1800700 1 -- 172.21.15.50:0/216425779 learned_addr learned my addr 172.21.15.50:0/216425779 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 -- 172.21.15.50:0/216425779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 msgr2=0x7fc5ac19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 0x7fc5ac19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 -- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac10b8a0 msgr2=0x7fc5ac1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac10b8a0 0x7fc5ac1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 -- 172.21.15.50:0/216425779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac1a1df0 con 0x7fc5ac10af50 2024-05-10T21:06:47.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5b0e00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 0x7fc5ac19d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc598000c00 tx=0x7fc59800f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.624+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598027070 con 0x7fc5ac10af50 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.625+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac1a2080 con 0x7fc5ac10af50 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.625+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5980054f0 con 0x7fc5ac10af50 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.625+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598004930 con 0x7fc5ac10af50 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.625+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a2560 con 0x7fc5ac10af50 2024-05-10T21:06:47.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.626+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac0512c0 con 0x7fc5ac10af50 2024-05-10T21:06:47.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.629+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc59802a070 con 0x7fc5ac10af50 2024-05-10T21:06:47.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.630+0000 7fc5aa000700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 0x7fc5940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:47.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.630+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc5980e2ed0 con 0x7fc5ac10af50 2024-05-10T21:06:47.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.631+0000 7fc5b1800700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 0x7fc5940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.631+0000 7fc5b1800700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 0x7fc5940b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009fd0 tx=0x7fc5a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:47.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.632+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc598031080 con 0x7fc5ac10af50 2024-05-10T21:06:47.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.869+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5ac19e280 con 0x7fc5940b2880 2024-05-10T21:06:47.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.890+0000 7fc5aa000700 1 -- 172.21.15.50:0/216425779 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258817 (secure 0 0 0) 0x7fc5ac19e280 con 0x7fc5940b2880 2024-05-10T21:06:47.898 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 msgr2=0x7fc5940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 0x7fc5940b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009fd0 tx=0x7fc5a000b040 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 msgr2=0x7fc5ac19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 0x7fc5ac19d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc598000c00 tx=0x7fc59800f950 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 shutdown_connections 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5940b2880 0x7fc5940b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5ac10b8a0 0x7fc5ac1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5ac106620 0x7fc5ac19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 --2- 172.21.15.50:0/216425779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ac10af50 0x7fc5ac19d4c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.897+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 >> 172.21.15.50:0/216425779 conn(0x7fc5ac102030 msgr2=0x7fc5ac10a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:47.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.898+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 shutdown_connections 2024-05-10T21:06:47.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:47.898+0000 7fc5b2c00700 1 -- 172.21.15.50:0/216425779 wait complete. 2024-05-10T21:06:47.901 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:06:48.708 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-05-10T21:06:47.627292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659203,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":825,"num_read_kb":513,"num_write":299,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374184,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567640,"statfs":{"total":1151940427776,"available":1151557263360,"internally_reserved":0,"allocated":12853248,"data_stored":7214925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005537"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578816+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:25.578816+0000","last_peered":"2024-05-10T21:06:25.578816+0000","last_clean":"2024-05-10T21:06:25.578816+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:25.578816+0000","last_undegraded":"2024-05-10T21:06:25.578816+0000","last_fullsized":"2024-05-10T21:06:25.578816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580291+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:25.580291+0000","last_peered":"2024-05-10T21:06:25.580291+0000","last_clean":"2024-05-10T21:06:25.580291+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:25.580291+0000","last_undegraded":"2024-05-10T21:06:25.580291+0000","last_fullsized":"2024-05-10T21:06:25.580291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581398+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:25.581398+0000","last_peered":"2024-05-10T21:06:25.581398+0000","last_clean":"2024-05-10T21:06:25.581398+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:25.581398+0000","last_undegraded":"2024-05-10T21:06:25.581398+0000","last_fullsized":"2024-05-10T21:06:25.581398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556625+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556625+0000","last_peered":"2024-05-10T21:06:25.556625+0000","last_clean":"2024-05-10T21:06:25.556625+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556625+0000","last_undegraded":"2024-05-10T21:06:25.556625+0000","last_fullsized":"2024-05-10T21:06:25.556625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.563869+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.563869+0000","last_peered":"2024-05-10T21:06:25.563869+0000","last_clean":"2024-05-10T21:06:25.563869+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.563869+0000","last_undegraded":"2024-05-10T21:06:25.563869+0000","last_fullsized":"2024-05-10T21:06:25.563869+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562418+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562418+0000","last_peered":"2024-05-10T21:06:25.562418+0000","last_clean":"2024-05-10T21:06:25.562418+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562418+0000","last_undegraded":"2024-05-10T21:06:25.562418+0000","last_fullsized":"2024-05-10T21:06:25.562418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557802+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.557802+0000","last_peered":"2024-05-10T21:06:25.557802+0000","last_clean":"2024-05-10T21:06:25.557802+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.557802+0000","last_undegraded":"2024-05-10T21:06:25.557802+0000","last_fullsized":"2024-05-10T21:06:25.557802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559486+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.559486+0000","last_peered":"2024-05-10T21:06:25.559486+0000","last_clean":"2024-05-10T21:06:25.559486+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.559486+0000","last_undegraded":"2024-05-10T21:06:25.559486+0000","last_fullsized":"2024-05-10T21:06:25.559486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567789+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567789+0000","last_peered":"2024-05-10T21:06:25.567789+0000","last_clean":"2024-05-10T21:06:25.567789+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567789+0000","last_undegraded":"2024-05-10T21:06:25.567789+0000","last_fullsized":"2024-05-10T21:06:25.567789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566659+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566659+0000","last_peered":"2024-05-10T21:06:25.566659+0000","last_clean":"2024-05-10T21:06:25.566659+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566659+0000","last_undegraded":"2024-05-10T21:06:25.566659+0000","last_fullsized":"2024-05-10T21:06:25.566659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565929+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.565929+0000","last_peered":"2024-05-10T21:06:25.565929+0000","last_clean":"2024-05-10T21:06:25.565929+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.565929+0000","last_undegraded":"2024-05-10T21:06:25.565929+0000","last_fullsized":"2024-05-10T21:06:25.565929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558019+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558019+0000","last_peered":"2024-05-10T21:06:25.558019+0000","last_clean":"2024-05-10T21:06:25.558019+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558019+0000","last_undegraded":"2024-05-10T21:06:25.558019+0000","last_fullsized":"2024-05-10T21:06:25.558019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576251+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576251+0000","last_peered":"2024-05-10T21:06:25.576251+0000","last_clean":"2024-05-10T21:06:25.576251+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576251+0000","last_undegraded":"2024-05-10T21:06:25.576251+0000","last_fullsized":"2024-05-10T21:06:25.576251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.184551+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.184551+0000","last_peered":"2024-05-10T21:04:57.184551+0000","last_clean":"2024-05-10T21:04:57.184551+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.184551+0000","last_undegraded":"2024-05-10T21:04:57.184551+0000","last_fullsized":"2024-05-10T21:04:57.184551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569551+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.569551+0000","last_peered":"2024-05-10T21:06:25.569551+0000","last_clean":"2024-05-10T21:06:25.569551+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.569551+0000","last_undegraded":"2024-05-10T21:06:25.569551+0000","last_fullsized":"2024-05-10T21:06:25.569551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.549962+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:04:57.549962+0000","last_peered":"2024-05-10T21:04:57.549962+0000","last_clean":"2024-05-10T21:04:57.549962+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:04:57.549962+0000","last_undegraded":"2024-05-10T21:04:57.549962+0000","last_fullsized":"2024-05-10T21:04:57.549962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581364+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:25.581364+0000","last_peered":"2024-05-10T21:06:25.581364+0000","last_clean":"2024-05-10T21:06:25.581364+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:25.581364+0000","last_undegraded":"2024-05-10T21:06:25.581364+0000","last_fullsized":"2024-05-10T21:06:25.581364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891580+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891580+0000","last_peered":"2024-05-10T21:04:56.891580+0000","last_clean":"2024-05-10T21:04:56.891580+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891580+0000","last_undegraded":"2024-05-10T21:04:56.891580+0000","last_fullsized":"2024-05-10T21:04:56.891580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572211+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572211+0000","last_peered":"2024-05-10T21:06:25.572211+0000","last_clean":"2024-05-10T21:06:25.572211+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572211+0000","last_undegraded":"2024-05-10T21:06:25.572211+0000","last_fullsized":"2024-05-10T21:06:25.572211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891333+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891333+0000","last_peered":"2024-05-10T21:04:56.891333+0000","last_clean":"2024-05-10T21:04:56.891333+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891333+0000","last_undegraded":"2024-05-10T21:04:56.891333+0000","last_fullsized":"2024-05-10T21:04:56.891333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.563861+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.563861+0000","last_peered":"2024-05-10T21:06:25.563861+0000","last_clean":"2024-05-10T21:06:25.563861+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.563861+0000","last_undegraded":"2024-05-10T21:06:25.563861+0000","last_fullsized":"2024-05-10T21:06:25.563861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:43.950543+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:06:43.950543+0000","last_peered":"2024-05-10T21:06:43.950543+0000","last_clean":"2024-05-10T21:06:43.950543+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:06:43.950543+0000","last_undegraded":"2024-05-10T21:06:43.950543+0000","last_fullsized":"2024-05-10T21:06:43.950543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555871+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.555871+0000","last_peered":"2024-05-10T21:06:25.555871+0000","last_clean":"2024-05-10T21:06:25.555871+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.555871+0000","last_undegraded":"2024-05-10T21:06:25.555871+0000","last_fullsized":"2024-05-10T21:06:25.555871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.885924+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.885924+0000","last_peered":"2024-05-10T21:04:56.885924+0000","last_clean":"2024-05-10T21:04:56.885924+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.885924+0000","last_undegraded":"2024-05-10T21:04:56.885924+0000","last_fullsized":"2024-05-10T21:04:56.885924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572513+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572513+0000","last_peered":"2024-05-10T21:06:25.572513+0000","last_clean":"2024-05-10T21:06:25.572513+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572513+0000","last_undegraded":"2024-05-10T21:06:25.572513+0000","last_fullsized":"2024-05-10T21:06:25.572513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574600+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574600+0000","last_peered":"2024-05-10T21:06:25.574600+0000","last_clean":"2024-05-10T21:06:25.574600+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574600+0000","last_undegraded":"2024-05-10T21:06:25.574600+0000","last_fullsized":"2024-05-10T21:06:25.574600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575469+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575469+0000","last_peered":"2024-05-10T21:06:25.575469+0000","last_clean":"2024-05-10T21:06:25.575469+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575469+0000","last_undegraded":"2024-05-10T21:06:25.575469+0000","last_fullsized":"2024-05-10T21:06:25.575469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:46.954183+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:06:46.954183+0000","last_peered":"2024-05-10T21:06:46.954183+0000","last_clean":"2024-05-10T21:06:46.954183+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:06:46.954183+0000","last_undegraded":"2024-05-10T21:06:46.954183+0000","last_fullsized":"2024-05-10T21:06:46.954183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568310+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568310+0000","last_peered":"2024-05-10T21:06:25.568310+0000","last_clean":"2024-05-10T21:06:25.568310+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568310+0000","last_undegraded":"2024-05-10T21:06:25.568310+0000","last_fullsized":"2024-05-10T21:06:25.568310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.580506+0000","last_peered":"2024-05-10T21:06:25.580506+0000","last_clean":"2024-05-10T21:06:25.580506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.580506+0000","last_undegraded":"2024-05-10T21:06:25.580506+0000","last_fullsized":"2024-05-10T21:06:25.580506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":315,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:46.954136+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:06:46.954136+0000","last_peered":"2024-05-10T21:06:46.954136+0000","last_clean":"2024-05-10T21:06:46.954136+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:06:46.954136+0000","last_undegraded":"2024-05-10T21:06:46.954136+0000","last_fullsized":"2024-05-10T21:06:46.954136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":254,"num_read_kb":190,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552519+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.552519+0000","last_peered":"2024-05-10T21:06:25.552519+0000","last_clean":"2024-05-10T21:06:25.552519+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.552519+0000","last_undegraded":"2024-05-10T21:06:25.552519+0000","last_fullsized":"2024-05-10T21:06:25.552519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:43.008449+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:06:43.008449+0000","last_peered":"2024-05-10T21:06:43.008449+0000","last_clean":"2024-05-10T21:06:43.008449+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:06:43.008449+0000","last_undegraded":"2024-05-10T21:06:43.008449+0000","last_fullsized":"2024-05-10T21:06:43.008449+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893043+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.893043+0000","last_peered":"2024-05-10T21:04:56.893043+0000","last_clean":"2024-05-10T21:04:56.893043+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.893043+0000","last_undegraded":"2024-05-10T21:04:56.893043+0000","last_fullsized":"2024-05-10T21:04:56.893043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576512+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:25.576512+0000","last_peered":"2024-05-10T21:06:25.576512+0000","last_clean":"2024-05-10T21:06:25.576512+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:25.576512+0000","last_undegraded":"2024-05-10T21:06:25.576512+0000","last_fullsized":"2024-05-10T21:06:25.576512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895673+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895673+0000","last_peered":"2024-05-10T21:04:56.895673+0000","last_clean":"2024-05-10T21:04:56.895673+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895673+0000","last_undegraded":"2024-05-10T21:04:56.895673+0000","last_fullsized":"2024-05-10T21:04:56.895673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578224+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:25.578224+0000","last_peered":"2024-05-10T21:06:25.578224+0000","last_clean":"2024-05-10T21:06:25.578224+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:25.578224+0000","last_undegraded":"2024-05-10T21:06:25.578224+0000","last_fullsized":"2024-05-10T21:06:25.578224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555147+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.555147+0000","last_peered":"2024-05-10T21:06:25.555147+0000","last_clean":"2024-05-10T21:06:25.555147+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.555147+0000","last_undegraded":"2024-05-10T21:06:25.555147+0000","last_fullsized":"2024-05-10T21:06:25.555147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:45.011372+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:06:45.011372+0000","last_peered":"2024-05-10T21:06:45.011372+0000","last_clean":"2024-05-10T21:06:45.011372+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:06:45.011372+0000","last_undegraded":"2024-05-10T21:06:45.011372+0000","last_fullsized":"2024-05-10T21:06:45.011372+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570231+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:25.570231+0000","last_peered":"2024-05-10T21:06:25.570231+0000","last_clean":"2024-05-10T21:06:25.570231+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:25.570231+0000","last_undegraded":"2024-05-10T21:06:25.570231+0000","last_fullsized":"2024-05-10T21:06:25.570231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331885+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331885+0000","last_peered":"2024-05-10T21:04:57.331885+0000","last_clean":"2024-05-10T21:04:57.331885+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331885+0000","last_undegraded":"2024-05-10T21:04:57.331885+0000","last_fullsized":"2024-05-10T21:04:57.331885+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895557+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:04:56.895557+0000","last_peered":"2024-05-10T21:04:56.895557+0000","last_clean":"2024-05-10T21:04:56.895557+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:04:56.895557+0000","last_undegraded":"2024-05-10T21:04:56.895557+0000","last_fullsized":"2024-05-10T21:04:56.895557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552134+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:25.552134+0000","last_peered":"2024-05-10T21:06:25.552134+0000","last_clean":"2024-05-10T21:06:25.552134+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:25.552134+0000","last_undegraded":"2024-05-10T21:06:25.552134+0000","last_fullsized":"2024-05-10T21:06:25.552134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895689+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:04:56.895689+0000","last_peered":"2024-05-10T21:04:56.895689+0000","last_clean":"2024-05-10T21:04:56.895689+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:04:56.895689+0000","last_undegraded":"2024-05-10T21:04:56.895689+0000","last_fullsized":"2024-05-10T21:04:56.895689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891596+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891596+0000","last_peered":"2024-05-10T21:04:56.891596+0000","last_clean":"2024-05-10T21:04:56.891596+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891596+0000","last_undegraded":"2024-05-10T21:04:56.891596+0000","last_fullsized":"2024-05-10T21:04:56.891596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562727+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:25.562727+0000","last_peered":"2024-05-10T21:06:25.562727+0000","last_clean":"2024-05-10T21:06:25.562727+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:25.562727+0000","last_undegraded":"2024-05-10T21:06:25.562727+0000","last_fullsized":"2024-05-10T21:06:25.562727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.968140+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:25.968140+0000","last_peered":"2024-05-10T21:06:25.968140+0000","last_clean":"2024-05-10T21:06:25.968140+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:25.968140+0000","last_undegraded":"2024-05-10T21:06:25.968140+0000","last_fullsized":"2024-05-10T21:06:25.968140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.340707+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.340707+0000","last_peered":"2024-05-10T21:04:57.340707+0000","last_clean":"2024-05-10T21:04:57.340707+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.340707+0000","last_undegraded":"2024-05-10T21:04:57.340707+0000","last_fullsized":"2024-05-10T21:04:57.340707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557980+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:25.557980+0000","last_peered":"2024-05-10T21:06:25.557980+0000","last_clean":"2024-05-10T21:06:25.557980+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:25.557980+0000","last_undegraded":"2024-05-10T21:06:25.557980+0000","last_fullsized":"2024-05-10T21:06:25.557980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552796+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:25.552796+0000","last_peered":"2024-05-10T21:06:25.552796+0000","last_clean":"2024-05-10T21:06:25.552796+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:25.552796+0000","last_undegraded":"2024-05-10T21:06:25.552796+0000","last_fullsized":"2024-05-10T21:06:25.552796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569414+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:25.569414+0000","last_peered":"2024-05-10T21:06:25.569414+0000","last_clean":"2024-05-10T21:06:25.569414+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:25.569414+0000","last_undegraded":"2024-05-10T21:06:25.569414+0000","last_fullsized":"2024-05-10T21:06:25.569414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579761+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:25.579761+0000","last_peered":"2024-05-10T21:06:25.579761+0000","last_clean":"2024-05-10T21:06:25.579761+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:25.579761+0000","last_undegraded":"2024-05-10T21:06:25.579761+0000","last_fullsized":"2024-05-10T21:06:25.579761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68601,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":749,"num_read_kb":449,"num_write":174,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841588,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1093497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037817,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":501484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":502140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822737,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:06:48.713 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T21:06:48.713 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T21:06:48.713 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T21:06:48.713 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T21:06:48.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:48 smithi050 ceph-mon[29546]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:48.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:48 smithi050 ceph-mon[29546]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:49.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:48 smithi092 ceph-mon[34415]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:49.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:48 smithi092 ceph-mon[34415]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:49.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:48 smithi142 ceph-mon[34846]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:49.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:48 smithi142 ceph-mon[34846]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:06:49.230 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:50.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:50 smithi050 ceph-mon[29546]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:51.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:50 smithi092 ceph-mon[34415]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:51.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:50 smithi142 ceph-mon[34846]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:51.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 -- 172.21.15.50:0/1159509722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8400764b0 msgr2=0x7ff840114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:51.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 --2- 172.21.15.50:0/1159509722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8400764b0 0x7ff840114e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff830009990 tx=0x7ff830009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 -- 172.21.15.50:0/1159509722 shutdown_connections 2024-05-10T21:06:51.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 --2- 172.21.15.50:0/1159509722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840115370 0x7ff840117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 --2- 172.21.15.50:0/1159509722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840075b60 0x7ff840075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 --2- 172.21.15.50:0/1159509722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8400764b0 0x7ff840114e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 -- 172.21.15.50:0/1159509722 >> 172.21.15.50:0/1159509722 conn(0x7ff840100040 msgr2=0x7ff840102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 -- 172.21.15.50:0/1159509722 shutdown_connections 2024-05-10T21:06:51.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.522+0000 7ff848800700 1 -- 172.21.15.50:0/1159509722 wait complete. 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.523+0000 7ff848800700 1 Processor -- start 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 -- start start 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 0x7ff8401a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff840115370 0x7ff8401a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff84011aac0 con 0x7ff840075b60 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff84011a940 con 0x7ff840115370 2024-05-10T21:06:51.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.524+0000 7ff848800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff84011ac40 con 0x7ff8400764b0 2024-05-10T21:06:51.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 0x7ff8401a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:51.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:51.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58298/0 (socket says 172.21.15.50:58298) 2024-05-10T21:06:51.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 -- 172.21.15.50:0/3272829914 learned_addr learned my addr 172.21.15.50:0/3272829914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff846800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff840115370 0x7ff8401a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 -- 172.21.15.50:0/3272829914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff840115370 msgr2=0x7ff8401a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff840115370 0x7ff8401a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 -- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 msgr2=0x7ff8401a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 0x7ff8401a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845400700 1 -- 172.21.15.50:0/3272829914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401a6320 con 0x7ff8400764b0 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.525+0000 7ff845e00700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 0x7ff8401a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:51.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff845400700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff830005910 tx=0x7ff83000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:51.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830027070 con 0x7ff8400764b0 2024-05-10T21:06:51.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff830022470 con 0x7ff8400764b0 2024-05-10T21:06:51.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830017650 con 0x7ff8400764b0 2024-05-10T21:06:51.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401a65b0 con 0x7ff8400764b0 2024-05-10T21:06:51.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.526+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a6a40 con 0x7ff8400764b0 2024-05-10T21:06:51.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.528+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400512c0 con 0x7ff8400764b0 2024-05-10T21:06:51.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.530+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff83002a070 con 0x7ff8400764b0 2024-05-10T21:06:51.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.532+0000 7ff83aa00700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:51.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.532+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff8300e3200 con 0x7ff8400764b0 2024-05-10T21:06:51.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.532+0000 7ff845e00700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:51.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.533+0000 7ff845e00700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 0x7ff8280b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff834005950 tx=0x7ff83400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:51.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.534+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff830013070 con 0x7ff8400764b0 2024-05-10T21:06:51.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.821+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff84002e5f0 con 0x7ff8400764b0 2024-05-10T21:06:51.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.822+0000 7ff83aa00700 1 -- 172.21.15.50:0/3272829914 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff8300aaf80 con 0x7ff8400764b0 2024-05-10T21:06:51.824 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:06:51.824 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T21:06:51.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 msgr2=0x7ff8280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 0x7ff8280b4e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff834005950 tx=0x7ff83400b410 comp rx=0 tx=0).stop 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 msgr2=0x7ff8401a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff830005910 tx=0x7ff83000b670 comp rx=0 tx=0).stop 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 shutdown_connections 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff8280b2900 0x7ff8280b4e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff840115370 0x7ff8401a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff840075b60 0x7ff8401a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 --2- 172.21.15.50:0/3272829914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8400764b0 0x7ff8401a19f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:51.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.826+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 >> 172.21.15.50:0/3272829914 conn(0x7ff840100040 msgr2=0x7ff840100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:51.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.827+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 shutdown_connections 2024-05-10T21:06:51.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:51.827+0000 7ff848800700 1 -- 172.21.15.50:0/3272829914 wait complete. 2024-05-10T21:06:52.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:51 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3272829914' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T21:06:52.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:51 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3272829914' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T21:06:52.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:51 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3272829914' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T21:06:52.744 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T21:06:52.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:06:53.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:52 smithi092 ceph-mon[34415]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:53.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:52 smithi142 ceph-mon[34846]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:53.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:52 smithi050 ceph-mon[29546]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:53.224 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:55.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:54 smithi092 ceph-mon[34415]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:55.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:54 smithi142 ceph-mon[34846]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:55.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:54 smithi050 ceph-mon[29546]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:06:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 -- 172.21.15.50:0/2236484186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 msgr2=0x7f0270113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 --2- 172.21.15.50:0/2236484186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f0270113350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 -- 172.21.15.50:0/2236484186 shutdown_connections 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 --2- 172.21.15.50:0/2236484186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 0x7f027010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 --2- 172.21.15.50:0/2236484186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f0270113350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 --2- 172.21.15.50:0/2236484186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f0270106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.428+0000 7f0275200700 1 -- 172.21.15.50:0/2236484186 >> 172.21.15.50:0/2236484186 conn(0x7f0270102050 msgr2=0x7f02701044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.429+0000 7f0275200700 1 -- 172.21.15.50:0/2236484186 shutdown_connections 2024-05-10T21:06:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.429+0000 7f0275200700 1 -- 172.21.15.50:0/2236484186 wait complete. 2024-05-10T21:06:55.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.430+0000 7f0275200700 1 Processor -- start 2024-05-10T21:06:55.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.430+0000 7f0275200700 1 -- start start 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f027019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 0x7f02701a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02701166e0 con 0x7f0270106570 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0270116560 con 0x7f027010c5b0 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0275200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0270116860 con 0x7f0270106ec0 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f026de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f0265400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f027019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f026de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33336/0 (socket says 172.21.15.50:33336) 2024-05-10T21:06:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f026de00700 1 -- 172.21.15.50:0/2791117721 learned_addr learned my addr 172.21.15.50:0/2791117721 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:55.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.431+0000 7f026e800700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 0x7f02701a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 -- 172.21.15.50:0/2791117721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 msgr2=0x7f027019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f027019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 -- 172.21.15.50:0/2791117721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 msgr2=0x7f02701a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 0x7f02701a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 -- 172.21.15.50:0/2791117721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02701a1e60 con 0x7f0270106570 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f0265400700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f027019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f026de00700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f026000e970 tx=0x7f026000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260004a30 con 0x7f0270106570 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026000cc90 con 0x7f0270106570 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02701a2150 con 0x7f0270106570 2024-05-10T21:06:55.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.432+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260005610 con 0x7f0270106570 2024-05-10T21:06:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.433+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02701a9a70 con 0x7f0270106570 2024-05-10T21:06:55.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.434+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02700512c0 con 0x7f0270106570 2024-05-10T21:06:55.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.436+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f026000c7e0 con 0x7f0270106570 2024-05-10T21:06:55.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.440+0000 7f0267400700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 0x7f02540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:55.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.441+0000 7f0265400700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 0x7f02540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.441+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f02600e2390 con 0x7f0270106570 2024-05-10T21:06:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.442+0000 7f0265400700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 0x7f02540b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0270105360 tx=0x7f025800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:55.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.443+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02600aa080 con 0x7f0270106570 2024-05-10T21:06:55.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.733+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f027019e2f0 con 0x7f0270106570 2024-05-10T21:06:55.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.735+0000 7f0267400700 1 -- 172.21.15.50:0/2791117721 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7f02600a9980 con 0x7f0270106570 2024-05-10T21:06:55.737 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:06:55.737 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:06:55.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.740+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 msgr2=0x7f02540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:55.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.740+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 0x7f02540b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0270105360 tx=0x7f025800b040 comp rx=0 tx=0).stop 2024-05-10T21:06:55.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.740+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 msgr2=0x7f027019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:55.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.740+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f026000e970 tx=0x7f026000ec80 comp rx=0 tx=0).stop 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 shutdown_connections 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f02540b2880 0x7f02540b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027010c5b0 0x7f02701a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0270106570 0x7f027019cf90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 --2- 172.21.15.50:0/2791117721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0270106ec0 0x7f027019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 >> 172.21.15.50:0/2791117721 conn(0x7f0270102050 msgr2=0x7f0270103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 shutdown_connections 2024-05-10T21:06:55.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:55.741+0000 7f0275200700 1 -- 172.21.15.50:0/2791117721 wait complete. 2024-05-10T21:06:55.744 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:06:56.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:55 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2791117721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:06:56.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:55 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2791117721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:06:56.149 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:55 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2791117721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:06:56.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:06:57.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:56 smithi092 ceph-mon[34415]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:57.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:56 smithi142 ceph-mon[34846]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:57.141 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:06:57.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:56 smithi050 ceph-mon[29546]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:06:58.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:58 smithi050 ceph-mon[29546]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:59.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:58 smithi092 ceph-mon[34415]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:58 smithi142 ceph-mon[34846]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:06:59.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 -- 172.21.15.50:0/2328519293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970076bf0 msgr2=0x7f4970077000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 --2- 172.21.15.50:0/2328519293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970076bf0 0x7f4970077000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4960009990 tx=0x7f4960009ca0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 -- 172.21.15.50:0/2328519293 shutdown_connections 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 --2- 172.21.15.50:0/2328519293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970110c00 0x7f4970113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 --2- 172.21.15.50:0/2328519293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970076bf0 0x7f4970077000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 --2- 172.21.15.50:0/2328519293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970077540 0x7f49701106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 -- 172.21.15.50:0/2328519293 >> 172.21.15.50:0/2328519293 conn(0x7f49700ffea0 msgr2=0x7f49701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.526+0000 7f4976600700 1 -- 172.21.15.50:0/2328519293 shutdown_connections 2024-05-10T21:06:59.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.527+0000 7f4976600700 1 -- 172.21.15.50:0/2328519293 wait complete. 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.528+0000 7f4976600700 1 Processor -- start 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.528+0000 7f4976600700 1 -- start start 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 0x7f4970104320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 0x7f4970104da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49701166d0 con 0x7f4970077540 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4970116550 con 0x7f4970110c00 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4976600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4970116850 con 0x7f4970076bf0 2024-05-10T21:06:59.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4966a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:59.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4966a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33354/0 (socket says 172.21.15.50:33354) 2024-05-10T21:06:59.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f4966a00700 1 -- 172.21.15.50:0/99088052 learned_addr learned my addr 172.21.15.50:0/99088052 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:06:59.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.529+0000 7f496f400700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 0x7f4970104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:59.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f496fe00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 0x7f4970104da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:59.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 -- 172.21.15.50:0/99088052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 msgr2=0x7f4970104320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:59.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 0x7f4970104320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 -- 172.21.15.50:0/99088052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 msgr2=0x7f4970104da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 0x7f4970104da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 -- 172.21.15.50:0/99088052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49701ad600 con 0x7f4970077540 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f496f400700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 0x7f4970104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f496fe00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 0x7f4970104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.530+0000 7f4966a00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f495800e970 tx=0x7f495800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.531+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800cbd0 con 0x7f4970077540 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.531+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f495800cd70 con 0x7f4970077540 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.531+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49701ad890 con 0x7f4970077540 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.531+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958005680 con 0x7f4970077540 2024-05-10T21:06:59.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.531+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701ade20 con 0x7f4970077540 2024-05-10T21:06:59.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.533+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4970078330 con 0x7f4970077540 2024-05-10T21:06:59.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.535+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4958004750 con 0x7f4970077540 2024-05-10T21:06:59.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.538+0000 7f496cc00700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 0x7f49540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:06:59.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.538+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f49580e2360 con 0x7f4970077540 2024-05-10T21:06:59.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.538+0000 7f496f400700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 0x7f49540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:06:59.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.539+0000 7f496f400700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 0x7f49540b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f496000b2e0 tx=0x7f4960005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:06:59.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.543+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49580aa0b0 con 0x7f4970077540 2024-05-10T21:06:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.831+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4970105880 con 0x7f4970077540 2024-05-10T21:06:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.832+0000 7f496cc00700 1 -- 172.21.15.50:0/99088052 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7f4970105880 con 0x7f4970077540 2024-05-10T21:06:59.834 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:06:59.835 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:06:59.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 msgr2=0x7f49540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 0x7f49540b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f496000b2e0 tx=0x7f4960005fd0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 msgr2=0x7f4970104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f495800e970 tx=0x7f495800ec80 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 shutdown_connections 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f49540b2900 0x7f49540b4e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4970110c00 0x7f4970104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4970076bf0 0x7f4970104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 --2- 172.21.15.50:0/99088052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4970077540 0x7f4970104860 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 >> 172.21.15.50:0/99088052 conn(0x7f49700ffea0 msgr2=0x7f49701004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 shutdown_connections 2024-05-10T21:06:59.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:06:59.837+0000 7f4976600700 1 -- 172.21.15.50:0/99088052 wait complete. 2024-05-10T21:06:59.840 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:07:00.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:06:59 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/99088052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:00.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:06:59 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/99088052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:00.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:06:59 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/99088052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:00.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:07:00.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:00 smithi050 ceph-mon[29546]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:01.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:00 smithi092 ceph-mon[34415]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:01.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:00 smithi142 ceph-mon[34846]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:03.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:02 smithi050 ceph-mon[29546]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:03.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:02 smithi142 ceph-mon[34846]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.316+0000 7fc582200700 1 -- 172.21.15.50:0/783951685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105dd0 msgr2=0x7fc57c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.316+0000 7fc582200700 1 --2- 172.21.15.50:0/783951685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105dd0 0x7fc57c10e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc564009990 tx=0x7fc564009ca0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 -- 172.21.15.50:0/783951685 shutdown_connections 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 --2- 172.21.15.50:0/783951685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105480 0x7fc57c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 --2- 172.21.15.50:0/783951685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 0x7fc57c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 --2- 172.21.15.50:0/783951685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105dd0 0x7fc57c10e320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 -- 172.21.15.50:0/783951685 >> 172.21.15.50:0/783951685 conn(0x7fc57c0fef30 msgr2=0x7fc57c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 -- 172.21.15.50:0/783951685 shutdown_connections 2024-05-10T21:07:03.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.317+0000 7fc582200700 1 -- 172.21.15.50:0/783951685 wait complete. 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.318+0000 7fc582200700 1 Processor -- start 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.318+0000 7fc582200700 1 -- start start 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 0x7fc57c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 0x7fc57c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 0x7fc57c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:03.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c114450 con 0x7fc57c105480 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c1142d0 con 0x7fc57c105dd0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc582200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c1145d0 con 0x7fc57c10e8f0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc57b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 0x7fc57c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc57b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 0x7fc57c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33368/0 (socket says 172.21.15.50:33368) 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc57b400700 1 -- 172.21.15.50:0/4238512047 learned_addr learned my addr 172.21.15.50:0/4238512047 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc572a00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 0x7fc57c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.319+0000 7fc57be00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 0x7fc57c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 -- 172.21.15.50:0/4238512047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 msgr2=0x7fc57c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 0x7fc57c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 -- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 msgr2=0x7fc57c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 0x7fc57c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 -- 172.21.15.50:0/4238512047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57c1a1eb0 con 0x7fc57c105dd0 2024-05-10T21:07:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc572a00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 0x7fc57c19d580 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc5640059d0 tx=0x7fc5640047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564027070 con 0x7fc57c105dd0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.320+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57c1a2140 con 0x7fc57c105dd0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.321+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc564022470 con 0x7fc57c105dd0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.321+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564017650 con 0x7fc57c105dd0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.321+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57c1a2640 con 0x7fc57c105dd0 2024-05-10T21:07:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.322+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57c0512c0 con 0x7fc57c105dd0 2024-05-10T21:07:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.324+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc56402a070 con 0x7fc57c105dd0 2024-05-10T21:07:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.326+0000 7fc578c00700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 0x7fc5600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.326+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc5640e3440 con 0x7fc57c105dd0 2024-05-10T21:07:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.326+0000 7fc57b400700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 0x7fc5600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.327+0000 7fc57b400700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 0x7fc5600b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009680 tx=0x7fc56c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.328+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc564032080 con 0x7fc57c105dd0 2024-05-10T21:07:03.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:02 smithi092 ceph-mon[34415]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.566+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc57c1a9ac0 con 0x7fc57c105dd0 2024-05-10T21:07:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.569+0000 7fc578c00700 1 -- 172.21.15.50:0/4238512047 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7fc57c1a2640 con 0x7fc57c105dd0 2024-05-10T21:07:03.571 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:03.571 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:07:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 msgr2=0x7fc5600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 0x7fc5600b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009680 tx=0x7fc56c009260 comp rx=0 tx=0).stop 2024-05-10T21:07:03.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 msgr2=0x7fc57c19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:03.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 0x7fc57c19d580 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc5640059d0 tx=0x7fc5640047b0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 shutdown_connections 2024-05-10T21:07:03.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5600b2880 0x7fc5600b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc57c10e8f0 0x7fc57c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57c105480 0x7fc57c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 --2- 172.21.15.50:0/4238512047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57c105dd0 0x7fc57c19d580 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:03.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.572+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 >> 172.21.15.50:0/4238512047 conn(0x7fc57c0fef30 msgr2=0x7fc57c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:03.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.573+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 shutdown_connections 2024-05-10T21:07:03.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:03.573+0000 7fc582200700 1 -- 172.21.15.50:0/4238512047 wait complete. 2024-05-10T21:07:04.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:03 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4238512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:04.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:03 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4238512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:04.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:03 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4238512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:04.488 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:07:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:05.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:04 smithi050 ceph-mon[29546]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:05.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:04 smithi092 ceph-mon[34415]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:05.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:04 smithi142 ceph-mon[34846]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:07.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.107+0000 7fe7c6800700 1 -- 172.21.15.50:0/2605198403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 msgr2=0x7fe7c0109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:07.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.107+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2605198403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c0109140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0099c0 tx=0x7fe7ac009cd0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 -- 172.21.15.50:0/2605198403 shutdown_connections 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2605198403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2605198403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 0x7fe7c0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2605198403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c0109140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 -- 172.21.15.50:0/2605198403 >> 172.21.15.50:0/2605198403 conn(0x7fe7c0079d10 msgr2=0x7fe7c007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 -- 172.21.15.50:0/2605198403 shutdown_connections 2024-05-10T21:07:07.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.108+0000 7fe7c6800700 1 -- 172.21.15.50:0/2605198403 wait complete. 2024-05-10T21:07:07.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:06 smithi050 ceph-mon[29546]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.109+0000 7fe7c6800700 1 Processor -- start 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.109+0000 7fe7c6800700 1 -- start start 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c01a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 0x7fe7c019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:07.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0114290 con 0x7fe7c0108d30 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0114110 con 0x7fe7c0104400 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c6800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0114410 con 0x7fe7c0109680 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c4a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c01a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c4a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c01a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60858/0 (socket says 172.21.15.50:60858) 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c4a00700 1 -- 172.21.15.50:0/2940211081 learned_addr learned my addr 172.21.15.50:0/2940211081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5e00700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 0x7fe7c019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 -- 172.21.15.50:0/2940211081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 msgr2=0x7fe7c019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 0x7fe7c019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 -- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 msgr2=0x7fe7c01a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c01a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.110+0000 7fe7c5400700 1 -- 172.21.15.50:0/2940211081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c019b260 con 0x7fe7c0104400 2024-05-10T21:07:07.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7c5400700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c01a09a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9f0 tx=0x7fe7b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:07.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400c9c0 con 0x7fe7c0104400 2024-05-10T21:07:07.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b400cb60 con 0x7fe7c0104400 2024-05-10T21:07:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4018680 con 0x7fe7c0104400 2024-05-10T21:07:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c019b550 con 0x7fe7c0104400 2024-05-10T21:07:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.111+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a7750 con 0x7fe7c0104400 2024-05-10T21:07:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.112+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a0005330 con 0x7fe7c0104400 2024-05-10T21:07:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.115+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe7b404aab0 con 0x7fe7c0104400 2024-05-10T21:07:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.117+0000 7fe7be000700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 0x7fe7a80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.117+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fe7b40e2390 con 0x7fe7c0104400 2024-05-10T21:07:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.117+0000 7fe7c4a00700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 0x7fe7a80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.118+0000 7fe7c4a00700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 0x7fe7a80b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7c007a310 tx=0x7fe7ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.119+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7b40aa080 con 0x7fe7c0104400 2024-05-10T21:07:07.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:06 smithi092 ceph-mon[34415]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:07.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:06 smithi142 ceph-mon[34846]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:07.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.407+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a0005620 con 0x7fe7c0104400 2024-05-10T21:07:07.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.409+0000 7fe7be000700 1 -- 172.21.15.50:0/2940211081 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7fe7b40b2100 con 0x7fe7c0104400 2024-05-10T21:07:07.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:07.411 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:07:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 msgr2=0x7fe7a80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 0x7fe7a80b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7c007a310 tx=0x7fe7ac01a040 comp rx=0 tx=0).stop 2024-05-10T21:07:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 msgr2=0x7fe7c01a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c01a09a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9f0 tx=0x7fe7b400ed00 comp rx=0 tx=0).stop 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 shutdown_connections 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7a80b2900 0x7fe7a80b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c0104400 0x7fe7c01a09a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7c0109680 0x7fe7c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 --2- 172.21.15.50:0/2940211081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c0108d30 0x7fe7c01a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.412+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 >> 172.21.15.50:0/2940211081 conn(0x7fe7c0079d10 msgr2=0x7fe7c0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:07.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.413+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 shutdown_connections 2024-05-10T21:07:07.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:07.413+0000 7fe7c6800700 1 -- 172.21.15.50:0/2940211081 wait complete. 2024-05-10T21:07:07.415 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:07:08.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:07 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2940211081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:08.282 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:07 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2940211081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:08.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T21:04:26.742871+0000', 'modified': '2024-05-10T21:04:50.607977+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24539, 'mds_1': 24520, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/1464775005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1464775005}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1464775005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/924361476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 924361476}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 924361476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6837/1752993045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1752993045}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1752993045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6839/3522449943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3522449943}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3522449943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.50:6841/908891204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6840', 'nonce': 908891204}, {'type': 'v1', 'addr': '172.21.15.50:6841', 'nonce': 908891204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6839/3200447854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3200447854}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3200447854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24539, 24502, 24520]} 2024-05-10T21:07:08.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T21:07:08.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T21:07:08.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:07 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2940211081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:08.804 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:09.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:08 smithi050 ceph-mon[29546]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:09.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:08 smithi092 ceph-mon[34415]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:09.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:08 smithi142 ceph-mon[34846]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 -- 172.21.15.50:0/3745803342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 msgr2=0x7f7b28110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3745803342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b28110470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b100098f0 tx=0x7f7b10009c00 comp rx=0 tx=0).stop 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 -- 172.21.15.50:0/3745803342 shutdown_connections 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3745803342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 0x7f7b28112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3745803342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b28076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3745803342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b28110470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 -- 172.21.15.50:0/3745803342 >> 172.21.15.50:0/3745803342 conn(0x7f7b28079d10 msgr2=0x7f7b2807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:10.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 -- 172.21.15.50:0/3745803342 shutdown_connections 2024-05-10T21:07:10.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.947+0000 7f7b27e00700 1 -- 172.21.15.50:0/3745803342 wait complete. 2024-05-10T21:07:10.950 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:10 smithi050 ceph-mon[29546]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:10.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.948+0000 7f7b27e00700 1 Processor -- start 2024-05-10T21:07:10.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.948+0000 7f7b27e00700 1 -- start start 2024-05-10T21:07:10.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b2819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 0x7f7b281a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28116520 con 0x7f7b28075d30 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281163a0 con 0x7f7b28076680 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281166a0 con 0x7f7b281109b0 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b26a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b26a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60880/0 (socket says 172.21.15.50:60880) 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b26a00700 1 -- 172.21.15.50:0/3864361821 learned_addr learned my addr 172.21.15.50:0/3864361821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b1e000700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b2819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.949+0000 7f7b27400700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 0x7f7b281a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 -- 172.21.15.50:0/3864361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 msgr2=0x7f7b281a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 0x7f7b281a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 -- 172.21.15.50:0/3864361821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 msgr2=0x7f7b2819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b2819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 -- 172.21.15.50:0/3864361821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281a1e00 con 0x7f7b28075d30 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b1e000700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b2819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800e9a0 tx=0x7f7b1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.950+0000 7f7b1fe00700 1 -- 172.21.15.50:0/3864361821 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800ca10 con 0x7f7b28075d30 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.951+0000 7f7b1fe00700 1 -- 172.21.15.50:0/3864361821 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b1800cbb0 con 0x7f7b28075d30 2024-05-10T21:07:10.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.951+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b281a20f0 con 0x7f7b28075d30 2024-05-10T21:07:10.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.951+0000 7f7b1fe00700 1 -- 172.21.15.50:0/3864361821 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18018680 con 0x7f7b28075d30 2024-05-10T21:07:10.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.951+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281a9a10 con 0x7f7b28075d30 2024-05-10T21:07:10.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.955+0000 7f7b1fe00700 1 -- 172.21.15.50:0/3864361821 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7b18010430 con 0x7f7b28075d30 2024-05-10T21:07:10.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 Processor -- start 2024-05-10T21:07:10.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 -- start start 2024-05-10T21:07:10.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b1fe00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 0x7f7b0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e47000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d689d0 con 0x5623b1e4ca30 2024-05-10T21:07:10.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d06710 con 0x5623b1cc8df0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.956+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d64990 con 0x5623b1e46bf0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b1fe00700 1 -- 172.21.15.50:0/3864361821 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7b180e2390 con 0x7f7b28075d30 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e47000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b27400700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 0x7f7b0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e47000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43172/0 (socket says 172.21.15.50:43172) 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b27400700 1 -- 172.21.15.50:0/2517725970 learned_addr learned my addr 172.21.15.50:0/2517725970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b1e000700 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.957+0000 7f7b26a00700 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 -- 172.21.15.50:0/2517725970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 msgr2=0x5623b1e47000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e47000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 -- 172.21.15.50:0/2517725970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 msgr2=0x5623b1e466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 -- 172.21.15.50:0/2517725970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b1cb5880 con 0x5623b1cc8df0 2024-05-10T21:07:10.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b27400700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 0x7f7b0c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b20008c60 tx=0x7f7b20006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:10.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b26a00700 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4c4f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000c00 tx=0x7f7b18082530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd7039a34ff2b6fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:10.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b17e00700 1 -- 172.21.15.50:0/2517725970 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18084440 con 0x5623b1cc8df0 2024-05-10T21:07:10.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b17e00700 1 -- 172.21.15.50:0/2517725970 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b180b2120 con 0x5623b1cc8df0 2024-05-10T21:07:10.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.958+0000 7f7b17e00700 1 -- 172.21.15.50:0/2517725970 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180add30 con 0x5623b1cc8df0 2024-05-10T21:07:10.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/2517725970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 msgr2=0x5623b1e4c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4c4f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000c00 tx=0x7f7b18082530 comp rx=0 tx=0).stop 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/2517725970 shutdown_connections 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4c4f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e47000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/2517725970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.959+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/2517725970 >> 172.21.15.50:0/2517725970 conn(0x5623b1d0df20 msgr2=0x5623b1e413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.960+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/2517725970 shutdown_connections 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.960+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/2517725970 wait complete. 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.960+0000 7f7b3c63bb80 1 -- start start 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.960+0000 7f7b3c63bb80 1 Processor -- start 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e50150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e50bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e53440 con 0x5623b1e4ca30 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5623b1ccead0 con 0x5623b1cc8df0 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e535c0 con 0x5623b1e46bf0 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b3c63bb80 10 client.0 _ll_register_callbacks cb 0x5623b1d0fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b27400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e50bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b1e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b1e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43188/0 (socket says 172.21.15.50:43188) 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b1e000700 1 -- 172.21.15.50:0/391224398 learned_addr learned my addr 172.21.15.50:0/391224398 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.961+0000 7f7b26a00700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e50150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 -- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 msgr2=0x5623b1e50690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 -- 172.21.15.50:0/391224398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 msgr2=0x5623b1e50bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e50bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 -- 172.21.15.50:0/391224398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b1d2e3d0 con 0x5623b1cc8df0 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b1e000700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b26a00700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e50150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b18082260 tx=0x7f7b180a9f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b515a7d06d406c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b14c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:07:10.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180c3060 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.962+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623b1e13170 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7b18082060 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7b180cc2c0 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 10 client.24887 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623b1e5dff0 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5623b1e13170 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7b180b2350 con 0x5623b1cc8df0 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 10 client.24887 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 10 client.24887 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 20 client.24887 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1e60110 0x5623b1e61580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.963+0000 7f7b3c63bb80 4 client.24887 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.964+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b1d763b0 con 0x5623b1e60110 2024-05-10T21:07:10.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.964+0000 7f7b1e000700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1e60110 0x5623b1e61580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.965+0000 7f7b1e000700 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1e60110 0x5623b1e61580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4dcb9711ab523f6 server_cookie=d66d6bde79d77641 in_seq=0 out_seq=0 2024-05-10T21:07:10.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.965+0000 7f7b14c00700 10 client.24887 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.965+0000 7f7b14c00700 1 -- 172.21.15.50:0/391224398 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623b1d763b0 con 0x5623b1e60110 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.965+0000 7f7b14c00700 10 client.24887 handle_command_reply: tid=0 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 client.24887 shutdown 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 msgr2=0x5623b1e50150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e50150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b18082260 tx=0x7f7b180a9f70 comp rx=0 tx=0).stop 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 shutdown_connections 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1e60110 0x5623b1e61580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e50150 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e50bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/391224398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e50690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 >> 172.21.15.50:0/391224398 conn(0x5623b1d0df20 msgr2=0x5623b1e456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:10.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b14c00700 0 client.24887 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:07:10.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 shutdown_connections 2024-05-10T21:07:10.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/391224398 wait complete. 2024-05-10T21:07:10.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.966+0000 7f7b3c63bb80 20 client.24887 trim_cache size 0 max 16384 2024-05-10T21:07:10.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.991+0000 7f7b3c63bb80 1 Processor -- start 2024-05-10T21:07:10.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 -- start start 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e5c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e5f8c0 con 0x5623b1e4ca30 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e5ff90 con 0x5623b1cc8df0 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e60660 con 0x5623b1e46bf0 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b27400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b1e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e5c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b26a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b27400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60898/0 (socket says 172.21.15.50:60898) 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.992+0000 7f7b27400700 1 -- 172.21.15.50:0/1552358825 learned_addr learned my addr 172.21.15.50:0/1552358825 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 -- 172.21.15.50:0/1552358825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 msgr2=0x5623b1e466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 -- 172.21.15.50:0/1552358825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 msgr2=0x5623b1e414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:10.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 -- 172.21.15.50:0/1552358825 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b1e3dbf0 con 0x5623b1e46bf0 2024-05-10T21:07:10.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b1e000700 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e5c860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009810 tx=0x7f7b1000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb35a215af753ba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:10.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.993+0000 7f7b17400700 1 -- 172.21.15.50:0/1552358825 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10005040 con 0x5623b1e46bf0 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.994+0000 7f7b17400700 1 -- 172.21.15.50:0/1552358825 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b10005690 con 0x5623b1e46bf0 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.994+0000 7f7b17400700 1 -- 172.21.15.50:0/1552358825 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000b5c0 con 0x5623b1e46bf0 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/1552358825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 msgr2=0x5623b1e5c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e5c860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009810 tx=0x7f7b1000bf60 comp rx=0 tx=0).stop 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/1552358825 shutdown_connections 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e4ca30 0x5623b1e414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/1552358825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e46bf0 0x5623b1e5c860 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.995+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/1552358825 >> 172.21.15.50:0/1552358825 conn(0x5623b1d0df20 msgr2=0x5623b1e49e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.996+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/1552358825 shutdown_connections 2024-05-10T21:07:10.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.996+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/1552358825 wait complete. 2024-05-10T21:07:10.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.996+0000 7f7b3c63bb80 1 -- start start 2024-05-10T21:07:10.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.996+0000 7f7b3c63bb80 1 Processor -- start 2024-05-10T21:07:10.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e46bf0 0x5623b1e4ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 0x5623b1e50480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e61400 con 0x5623b1e46bf0 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e60d30 con 0x5623b1cc8df0 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5623b1e61ad0 con 0x5623b1e4ca30 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b3c63bb80 10 client.0 _ll_register_callbacks cb 0x5623b1d0fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b26a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b1e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e46bf0 0x5623b1e4ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b26a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55190/0 (socket says 172.21.15.50:55190) 2024-05-10T21:07:10.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b26a00700 1 -- 172.21.15.50:0/3849038819 learned_addr learned my addr 172.21.15.50:0/3849038819 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.997+0000 7f7b27400700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 0x5623b1e50480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 -- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 msgr2=0x5623b1e50480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 0x5623b1e50480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 -- 172.21.15.50:0/3849038819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e46bf0 msgr2=0x5623b1e4ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e46bf0 0x5623b1e4ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 -- 172.21.15.50:0/3849038819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b1e699f0 con 0x5623b1cc8df0 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b27400700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 0x5623b1e50480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:07:11.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b26a00700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b180a9680 tx=0x7f7b180a9880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb7056192ab862c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b14c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180c3030 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7b180a5f70 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.998+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b10005690 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7b180a5840 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 10 client.24893 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5623b1e6ac50 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b10005690 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7b180a8be0 con 0x5623b1cc8df0 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 10 client.24893 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 10 client.24893 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:07:11.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 20 client.24893 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:11.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1eccde0 0x5623b1ece250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:11.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 4 client.24893 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:07:11.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:10.999+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "status"}) -- 0x5623b1d70450 con 0x5623b1eccde0 2024-05-10T21:07:11.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.000+0000 7f7b1e000700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1eccde0 0x5623b1ece250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.000+0000 7f7b1e000700 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1eccde0 0x5623b1ece250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3491eb855db41e50 server_cookie=d8e390bb24db37aa in_seq=0 out_seq=0 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.000+0000 7f7b14c00700 10 client.24893 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b14c00700 1 -- 172.21.15.50:0/3849038819 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5623b1d70450 con 0x5623b1eccde0 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b14c00700 10 client.24893 handle_command_reply: tid=0 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 client.24893 shutdown 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 msgr2=0x5623b1e4fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b180a9680 tx=0x7f7b180a9880 comp rx=0 tx=0).stop 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 shutdown_connections 2024-05-10T21:07:11.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5623b1eccde0 0x5623b1ece250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5623b1cc8df0 0x5623b1e4fa00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5623b1e4ca30 0x5623b1e50480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b14c00700 0 client.24893 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 --2- 172.21.15.50:0/3849038819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b1e46bf0 0x5623b1e4ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 >> 172.21.15.50:0/3849038819 conn(0x5623b1d0df20 msgr2=0x5623b1e3b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 shutdown_connections 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 1 -- 172.21.15.50:0/3849038819 wait complete. 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.001+0000 7f7b3c63bb80 20 client.24893 trim_cache size 0 max 16384 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:07:11.005 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24502, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 147.19990197000001, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 198.12145106700001 2024-05-10T21:07:11.006 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 msgr2=0x7f7b0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 0x7f7b0c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b20008c60 tx=0x7f7b20006040 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 msgr2=0x7f7b2819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800e9a0 tx=0x7f7b1800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 shutdown_connections 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7b0c0b2880 0x7f7b0c0b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b281109b0 0x7f7b281a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b28075d30 0x7f7b2819cf30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 --2- 172.21.15.50:0/3864361821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b28076680 0x7f7b2819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:11.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 >> 172.21.15.50:0/3864361821 conn(0x7f7b28079d10 msgr2=0x7f7b28100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:11.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 shutdown_connections 2024-05-10T21:07:11.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:11.002+0000 7f7b27e00700 1 -- 172.21.15.50:0/3864361821 wait complete. 2024-05-10T21:07:11.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:10 smithi092 ceph-mon[34415]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:11.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:10 smithi142 ceph-mon[34846]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:12.004 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:11 smithi050 ceph-mon[29546]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:12.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.a status 2024-05-10T21:07:12.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:11 smithi092 ceph-mon[34415]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:12.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:11 smithi142 ceph-mon[34846]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:12.458 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:14.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 -- 172.21.15.50:0/3534279446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4101045f0 msgr2=0x7ff410104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 --2- 172.21.15.50:0/3534279446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4101045f0 0x7ff410104a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4040099c0 tx=0x7ff404009cd0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 -- 172.21.15.50:0/3534279446 shutdown_connections 2024-05-10T21:07:14.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 --2- 172.21.15.50:0/3534279446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 0x7ff4101111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 --2- 172.21.15.50:0/3534279446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4101045f0 0x7ff410104a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 --2- 172.21.15.50:0/3534279446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff410108f20 0x7ff410109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 -- 172.21.15.50:0/3534279446 >> 172.21.15.50:0/3534279446 conn(0x7ff410100040 msgr2=0x7ff410102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 -- 172.21.15.50:0/3534279446 shutdown_connections 2024-05-10T21:07:14.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.740+0000 7ff418800700 1 -- 172.21.15.50:0/3534279446 wait complete. 2024-05-10T21:07:14.743 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:14 smithi050 ceph-mon[29546]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.742+0000 7ff418800700 1 Processor -- start 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.742+0000 7ff418800700 1 -- start start 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.742+0000 7ff418800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff418800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff410108f20 0x7ff41019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff418800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 0x7ff41019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff418800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101144a0 con 0x7ff410108f20 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff418800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff410114320 con 0x7ff410109870 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff418800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff410114620 con 0x7ff4101045f0 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41990/0 (socket says 172.21.15.50:41990) 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 -- 172.21.15.50:0/192694127 learned_addr learned my addr 172.21.15.50:0/192694127 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415400700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff410108f20 0x7ff41019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff416800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 0x7ff41019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 -- 172.21.15.50:0/192694127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 msgr2=0x7ff41019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 0x7ff41019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 -- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff410108f20 msgr2=0x7ff41019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff410108f20 0x7ff41019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.743+0000 7ff415e00700 1 -- 172.21.15.50:0/192694127 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41019fce0 con 0x7ff4101045f0 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff415e00700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff404006010 tx=0x7ff404005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff40aa00700 1 -- 172.21.15.50:0/192694127 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404026070 con 0x7ff4101045f0 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41019ff70 con 0x7ff4101045f0 2024-05-10T21:07:14.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101a78f0 con 0x7ff4101045f0 2024-05-10T21:07:14.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff40aa00700 1 -- 172.21.15.50:0/192694127 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff404005680 con 0x7ff4101045f0 2024-05-10T21:07:14.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.744+0000 7ff40aa00700 1 -- 172.21.15.50:0/192694127 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400fbb0 con 0x7ff4101045f0 2024-05-10T21:07:14.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.748+0000 7ff40aa00700 1 -- 172.21.15.50:0/192694127 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff404029070 con 0x7ff4101045f0 2024-05-10T21:07:14.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.749+0000 7ff42739db80 1 Processor -- start 2024-05-10T21:07:14.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.749+0000 7ff40aa00700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 0x7ff3f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.749+0000 7ff42739db80 1 -- start start 2024-05-10T21:07:14.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff40aa00700 1 -- 172.21.15.50:0/192694127 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff4040e2a90 con 0x7ff4101045f0 2024-05-10T21:07:14.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff416800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 0x7ff3f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa0dc9d0 con 0x55e0aa1bf070 2024-05-10T21:07:14.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa07a710 con 0x55e0aa1c3780 2024-05-10T21:07:14.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff42739db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa0d8990 con 0x55e0aa03cdf0 2024-05-10T21:07:14.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff415400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff415400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59596/0 (socket says 172.21.15.50:59596) 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff415400700 1 -- 172.21.15.50:0/3739312822 learned_addr learned my addr 172.21.15.50:0/3739312822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff416800700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.750+0000 7ff415e00700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff416800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 0x7ff3f80b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00f590 tx=0x7ff40c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 -- 172.21.15.50:0/3739312822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 msgr2=0x55e0aa1c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 -- 172.21.15.50:0/3739312822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 msgr2=0x55e0aa1c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 -- 172.21.15.50:0/3739312822 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0aa029880 con 0x55e0aa03cdf0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415400700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff415e00700 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1beb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff404009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13fb84f37bbd5f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.751+0000 7ff3ff400700 1 -- 172.21.15.50:0/3739312822 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040aa110 con 0x55e0aa03cdf0 2024-05-10T21:07:14.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.752+0000 7ff3ff400700 1 -- 172.21.15.50:0/3739312822 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4040ae4c0 con 0x55e0aa03cdf0 2024-05-10T21:07:14.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.752+0000 7ff3ff400700 1 -- 172.21.15.50:0/3739312822 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040b2cb0 con 0x55e0aa03cdf0 2024-05-10T21:07:14.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.752+0000 7ff42739db80 1 -- 172.21.15.50:0/3739312822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 msgr2=0x55e0aa1beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.752+0000 7ff42739db80 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1beb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff404009ee0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 -- 172.21.15.50:0/3739312822 shutdown_connections 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1beb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 --2- 172.21.15.50:0/3739312822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1bf070 0x55e0aa1c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 -- 172.21.15.50:0/3739312822 >> 172.21.15.50:0/3739312822 conn(0x55e0aa081f20 msgr2=0x55e0aa1b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 -- 172.21.15.50:0/3739312822 shutdown_connections 2024-05-10T21:07:14.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 -- 172.21.15.50:0/3739312822 wait complete. 2024-05-10T21:07:14.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.753+0000 7ff42739db80 1 -- start start 2024-05-10T21:07:14.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 Processor -- start 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1ba460 con 0x55e0aa03cdf0 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa042ad0 con 0x55e0aa1c3780 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1ba5e0 con 0x55e0aa1bf070 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.754+0000 7ff42739db80 10 client.0 _ll_register_callbacks cb 0x55e0aa083c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42010/0 (socket says 172.21.15.50:42010) 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 -- 172.21.15.50:0/2885510199 learned_addr learned my addr 172.21.15.50:0/2885510199 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff416800700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415e00700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 -- 172.21.15.50:0/2885510199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 msgr2=0x55e0aa1bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 -- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 msgr2=0x55e0aa1c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.755+0000 7ff415400700 1 -- 172.21.15.50:0/2885510199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0aa0a23d0 con 0x55e0aa1bf070 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff415400700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff400004600 tx=0x7ff400003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe966a2f8fd51928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff3f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:07:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400027060 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4040ae4c0 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff40000f5d0 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff40000f7d0 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff42739db80 10 client.24895 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.756+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e0aa1d1bf0 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4040ae4c0 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff40002da40 con 0x55e0aa1bf070 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 10 client.24895 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 10 client.24895 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 20 client.24895 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa1d4230 0x55e0aa1d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 4 client.24895 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:07:14.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.757+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0aa187170 con 0x55e0aa1d4230 2024-05-10T21:07:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.758+0000 7ff415e00700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa1d4230 0x55e0aa1d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.758+0000 7ff415e00700 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa1d4230 0x55e0aa1d56a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df0f3cf0469a2450 server_cookie=e703564b7c23536c in_seq=0 out_seq=0 2024-05-10T21:07:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.758+0000 7ff3f3e00700 10 client.24895 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.759+0000 7ff3f3e00700 1 -- 172.21.15.50:0/2885510199 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0aa187170 con 0x55e0aa1d4230 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.759+0000 7ff3f3e00700 10 client.24895 handle_command_reply: tid=0 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.759+0000 7ff42739db80 1 client.24895 shutdown 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 msgr2=0x55e0aa1ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff400004600 tx=0x7ff400003960 comp rx=0 tx=0).stop 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 shutdown_connections 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa1bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa1d4230 0x55e0aa1d56a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 --2- 172.21.15.50:0/2885510199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1ca450 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff3f3e00700 0 client.24895 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 >> 172.21.15.50:0/2885510199 conn(0x55e0aa081f20 msgr2=0x55e0aa1b9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 shutdown_connections 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 1 -- 172.21.15.50:0/2885510199 wait complete. 2024-05-10T21:07:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.760+0000 7ff42739db80 20 client.24895 trim_cache size 0 max 16384 2024-05-10T21:07:14.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.784+0000 7ff42739db80 1 Processor -- start 2024-05-10T21:07:14.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.784+0000 7ff42739db80 1 -- start start 2024-05-10T21:07:14.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 0x55e0aa1df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1c3780 0x55e0aa1b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d39e0 con 0x55e0aa1c3780 2024-05-10T21:07:14.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d40b0 con 0x55e0aa1bf070 2024-05-10T21:07:14.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff42739db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d4780 con 0x55e0aa03cdf0 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff415e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff416800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1c3780 0x55e0aa1b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff415400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 0x55e0aa1df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff415e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42014/0 (socket says 172.21.15.50:42014) 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.785+0000 7ff415e00700 1 -- 172.21.15.50:0/1122627768 learned_addr learned my addr 172.21.15.50:0/1122627768 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 -- 172.21.15.50:0/1122627768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 msgr2=0x55e0aa1df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 0x55e0aa1df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 -- 172.21.15.50:0/1122627768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1c3780 msgr2=0x55e0aa1b9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1c3780 0x55e0aa1b9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 -- 172.21.15.50:0/1122627768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0aa1bbc80 con 0x55e0aa03cdf0 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415400700 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 0x55e0aa1df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.786+0000 7ff415e00700 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff4040a6100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3668f619f0472ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.787+0000 7ff3fea00700 1 -- 172.21.15.50:0/1122627768 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400f460 con 0x55e0aa03cdf0 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.787+0000 7ff3fea00700 1 -- 172.21.15.50:0/1122627768 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4040ae130 con 0x55e0aa03cdf0 2024-05-10T21:07:14.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.787+0000 7ff3fea00700 1 -- 172.21.15.50:0/1122627768 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040b2cb0 con 0x55e0aa03cdf0 2024-05-10T21:07:14.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.787+0000 7ff42739db80 1 -- 172.21.15.50:0/1122627768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 msgr2=0x55e0aa1deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.787+0000 7ff42739db80 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff4040a6100 comp rx=0 tx=0).stop 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 -- 172.21.15.50:0/1122627768 shutdown_connections 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa1c3780 0x55e0aa1b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1deb00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 --2- 172.21.15.50:0/1122627768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1bf070 0x55e0aa1df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 -- 172.21.15.50:0/1122627768 >> 172.21.15.50:0/1122627768 conn(0x55e0aa081f20 msgr2=0x55e0aa1bcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 -- 172.21.15.50:0/1122627768 shutdown_connections 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 -- 172.21.15.50:0/1122627768 wait complete. 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 -- start start 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.788+0000 7ff42739db80 1 Processor -- start 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa23e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d5520 con 0x55e0aa03cdf0 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d4e50 con 0x55e0aa1c3780 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa1d5bf0 con 0x55e0aa1bf070 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff42739db80 10 client.0 _ll_register_callbacks cb 0x55e0aa083c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff415e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff415400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff415e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59610/0 (socket says 172.21.15.50:59610) 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.789+0000 7ff415e00700 1 -- 172.21.15.50:0/3889305663 learned_addr learned my addr 172.21.15.50:0/3889305663 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:14.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff416800700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa23e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 -- 172.21.15.50:0/3889305663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 msgr2=0x55e0aa23e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa23e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 -- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 msgr2=0x55e0aa1cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 -- 172.21.15.50:0/3889305663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0aa1cd140 con 0x55e0aa1bf070 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415e00700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff415400700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1cc860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff40002de30 tx=0x7ff400009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3596dd6a30ea3a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.790+0000 7ff3f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400032040 con 0x55e0aa1bf070 2024-05-10T21:07:14.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff400005070 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4040ae130 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff400005290 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff42739db80 10 client.24901 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e0aa23fbf0 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4040ae130 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff400035070 con 0x55e0aa1bf070 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.791+0000 7ff42739db80 10 client.24901 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff42739db80 10 client.24901 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff42739db80 20 client.24901 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa242060 0x55e0aa2434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff42739db80 4 client.24901 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "status"}) -- 0x55e0aa0ea3b0 con 0x55e0aa242060 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff415e00700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa242060 0x55e0aa2434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:14.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff415e00700 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa242060 0x55e0aa2434d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=681f22d607d546e8 server_cookie=5247961dea8419ab in_seq=0 out_seq=0 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.792+0000 7ff3f3e00700 10 client.24901 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff3f3e00700 1 -- 172.21.15.50:0/3889305663 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e0aa0ea3b0 con 0x55e0aa242060 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff3f3e00700 10 client.24901 handle_command_reply: tid=0 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 client.24901 shutdown 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 msgr2=0x55e0aa1cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1cc860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff40002de30 tx=0x7ff400009c40 comp rx=0 tx=0).stop 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 shutdown_connections 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0aa1c3780 0x55e0aa23e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55e0aa242060 0x55e0aa2434d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0aa03cdf0 0x55e0aa1cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 --2- 172.21.15.50:0/3889305663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0aa1bf070 0x55e0aa1cc860 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 >> 172.21.15.50:0/3889305663 conn(0x55e0aa081f20 msgr2=0x55e0aa1b9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.793+0000 7ff3f3e00700 0 client.24901 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:07:14.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.794+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 shutdown_connections 2024-05-10T21:07:14.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.794+0000 7ff42739db80 1 -- 172.21.15.50:0/3889305663 wait complete. 2024-05-10T21:07:14.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.794+0000 7ff42739db80 20 client.24901 trim_cache size 0 max 16384 2024-05-10T21:07:14.798 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24520, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 152.001479101, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 192.951819632 2024-05-10T21:07:14.799 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 msgr2=0x7ff3f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 0x7ff3f80b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00f590 tx=0x7ff40c00b040 comp rx=0 tx=0).stop 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 msgr2=0x7ff41019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff404006010 tx=0x7ff404005280 comp rx=0 tx=0).stop 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 shutdown_connections 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff3f80b2880 0x7ff3f80b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff410109870 0x7ff41019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4101045f0 0x7ff41019ae10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 --2- 172.21.15.50:0/192694127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff410108f20 0x7ff41019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:14.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 >> 172.21.15.50:0/192694127 conn(0x7ff410100040 msgr2=0x7ff41010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:14.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 shutdown_connections 2024-05-10T21:07:14.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:14.795+0000 7ff418800700 1 -- 172.21.15.50:0/192694127 wait complete. 2024-05-10T21:07:15.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:14 smithi092 ceph-mon[34415]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:15.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:14 smithi142 ceph-mon[34846]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:15.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T21:07:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:16.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:16 smithi050 ceph-mon[29546]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:17.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:16 smithi092 ceph-mon[34415]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:17.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:16 smithi142 ceph-mon[34846]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 -- 172.21.15.50:0/548184080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558075b60 msgr2=0x7fb558075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 --2- 172.21.15.50:0/548184080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558075b60 0x7fb558075f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb5480098f0 tx=0x7fb548009c00 comp rx=0 tx=0).stop 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 -- 172.21.15.50:0/548184080 shutdown_connections 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 --2- 172.21.15.50:0/548184080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558075b60 0x7fb558075f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 --2- 172.21.15.50:0/548184080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb558115370 0x7fb558117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 --2- 172.21.15.50:0/548184080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5580764b0 0x7fb558114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 -- 172.21.15.50:0/548184080 >> 172.21.15.50:0/548184080 conn(0x7fb558100040 msgr2=0x7fb558102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 -- 172.21.15.50:0/548184080 shutdown_connections 2024-05-10T21:07:18.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.507+0000 7fb55ea00700 1 -- 172.21.15.50:0/548184080 wait complete. 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.508+0000 7fb55ea00700 1 Processor -- start 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 -- start start 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 0x7fb5581a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 0x7fb5581a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb55811aa70 con 0x7fb558075b60 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb55811a8f0 con 0x7fb558115370 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb55ea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb55811abf0 con 0x7fb5580764b0 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59620/0 (socket says 172.21.15.50:59620) 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb557e00700 1 -- 172.21.15.50:0/265798149 learned_addr learned my addr 172.21.15.50:0/265798149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.509+0000 7fb54f400700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 0x7fb5581a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb55ca00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 0x7fb5581a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 -- 172.21.15.50:0/265798149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 msgr2=0x7fb5581a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 0x7fb5581a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 -- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 msgr2=0x7fb5581a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 -- 172.21.15.50:0/265798149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581a62d0 con 0x7fb5580764b0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb557e00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb55ca00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 0x7fb5581a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.510+0000 7fb54f400700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 0x7fb5581a19a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9f0 tx=0x7fb54000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.511+0000 7fb555600700 1 -- 172.21.15.50:0/265798149 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54000cb70 con 0x7fb5580764b0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.511+0000 7fb555600700 1 -- 172.21.15.50:0/265798149 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54000ee90 con 0x7fb5580764b0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.511+0000 7fb555600700 1 -- 172.21.15.50:0/265798149 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540003d40 con 0x7fb5580764b0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.511+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5581a65c0 con 0x7fb5580764b0 2024-05-10T21:07:18.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.511+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581adee0 con 0x7fb5580764b0 2024-05-10T21:07:18.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.515+0000 7fb555600700 1 -- 172.21.15.50:0/265798149 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb540003ee0 con 0x7fb5580764b0 2024-05-10T21:07:18.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 Processor -- start 2024-05-10T21:07:18.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 -- start start 2024-05-10T21:07:18.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48217e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 0x55a482178460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a48209a9d0 con 0x55a48217e7e0 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a482038710 con 0x55a481ffadf0 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.516+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a482096990 con 0x55a4821789a0 2024-05-10T21:07:18.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb555600700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 0x7fb53c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 0x55a482178460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb55ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48217e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb55ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48217e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40094/0 (socket says 172.21.15.50:40094) 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 0x55a482178460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59636/0 (socket says 172.21.15.50:59636) 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42040/0 (socket says 172.21.15.50:42040) 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb55ca00700 1 -- 172.21.15.50:0/2269066818 learned_addr learned my addr 172.21.15.50:0/2269066818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb55ca00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 0x7fb53c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 -- 172.21.15.50:0/2269066818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 msgr2=0x55a48217e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48217e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 -- 172.21.15.50:0/2269066818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 msgr2=0x55a482178460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 0x55a482178460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb54f400700 1 -- 172.21.15.50:0/2269066818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a481fe7880 con 0x55a4821789a0 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.517+0000 7fb555600700 1 -- 172.21.15.50:0/265798149 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb5400e2290 con 0x7fb5580764b0 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.518+0000 7fb54f400700 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5400a9580 tx=0x7fb5400a9250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e854849069966a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.518+0000 7fb54d600700 1 -- 172.21.15.50:0/2269066818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540056ce0 con 0x55a4821789a0 2024-05-10T21:07:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.518+0000 7fb55ca00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 0x7fb53c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb55000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.518+0000 7fb54d600700 1 -- 172.21.15.50:0/2269066818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb540059940 con 0x55a4821789a0 2024-05-10T21:07:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.518+0000 7fb54d600700 1 -- 172.21.15.50:0/2269066818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400a9880 con 0x55a4821789a0 2024-05-10T21:07:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2269066818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 msgr2=0x55a482178db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5400a9580 tx=0x7fb5400a9250 comp rx=0 tx=0).stop 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2269066818 shutdown_connections 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48217e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4821789a0 0x55a482178db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2269066818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a48217e7e0 0x55a482178460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2269066818 >> 172.21.15.50:0/2269066818 conn(0x55a48203ff20 msgr2=0x55a482173280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2269066818 shutdown_connections 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.519+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2269066818 wait complete. 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.520+0000 7fb56d4d1b80 1 -- start start 2024-05-10T21:07:18.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.520+0000 7fb56d4d1b80 1 Processor -- start 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4821789a0 0x55a4821824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a482182a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a482185200 con 0x55a481ffadf0 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a482000ad0 con 0x55a4821789a0 2024-05-10T21:07:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a482185380 con 0x55a48217e7e0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb56d4d1b80 10 client.0 _ll_register_callbacks cb 0x55a482041c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb557e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59644/0 (socket says 172.21.15.50:59644) 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb557e00700 1 -- 172.21.15.50:0/3882031028 learned_addr learned my addr 172.21.15.50:0/3882031028 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.521+0000 7fb54f400700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4821789a0 0x55a4821824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb55ca00700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a482182a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 -- 172.21.15.50:0/3882031028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 msgr2=0x55a482182a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a482182a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 -- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4821789a0 msgr2=0x55a4821824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4821789a0 0x55a4821824e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 -- 172.21.15.50:0/3882031028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4820603d0 con 0x55a481ffadf0 2024-05-10T21:07:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb557e00700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5480098c0 tx=0x7fb5480095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289d613a466a076 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.522+0000 7fb546000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548027020 con 0x55a481ffadf0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb5480054d0 con 0x55a481ffadf0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb540059940 con 0x55a481ffadf0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb548003680 con 0x55a481ffadf0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb56d4d1b80 10 client.14799 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a48218fd40 con 0x55a481ffadf0 2024-05-10T21:07:18.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.523+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb540059940 con 0x55a481ffadf0 2024-05-10T21:07:18.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb54800b990 con 0x55a481ffadf0 2024-05-10T21:07:18.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 10 client.14799 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:18.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 10 client.14799 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:07:18.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 20 client.14799 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821923f0 0x55a482193860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 4 client.14799 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a482145170 con 0x55a4821923f0 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.524+0000 7fb54f400700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821923f0 0x55a482193860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.525+0000 7fb54f400700 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821923f0 0x55a482193860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3267c0c8a9a50a75 server_cookie=7bb9e4336384188b in_seq=0 out_seq=0 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.525+0000 7fb546000700 10 client.14799 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb546000700 1 -- 172.21.15.50:0/3882031028 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a482145170 con 0x55a4821923f0 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb546000700 10 client.14799 handle_command_reply: tid=0 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 client.14799 shutdown 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 msgr2=0x55a482181fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5480098c0 tx=0x7fb5480095a0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 shutdown_connections 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821923f0 0x55a482193860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a482182a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a481ffadf0 0x55a482181fa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/3882031028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4821789a0 0x55a4821824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 >> 172.21.15.50:0/3882031028 conn(0x55a48203ff20 msgr2=0x55a482177450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb546000700 0 client.14799 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 shutdown_connections 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.526+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/3882031028 wait complete. 2024-05-10T21:07:18.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.527+0000 7fb56d4d1b80 20 client.14799 trim_cache size 0 max 16384 2024-05-10T21:07:18.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 Processor -- start 2024-05-10T21:07:18.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 -- start start 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a482178460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a482191ba0 con 0x55a4821789a0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a482192270 con 0x55a481ffadf0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.550+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a482192940 con 0x55a48217e7e0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59654/0 (socket says 172.21.15.50:59654) 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 -- 172.21.15.50:0/1418702373 learned_addr learned my addr 172.21.15.50:0/1418702373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb557e00700 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a482178460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb55ca00700 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 -- 172.21.15.50:0/1418702373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 msgr2=0x55a48218e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 -- 172.21.15.50:0/1418702373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 msgr2=0x55a482178460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a482178460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.551+0000 7fb54f400700 1 -- 172.21.15.50:0/1418702373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a482174b20 con 0x55a4821789a0 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.552+0000 7fb54f400700 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9c0 tx=0x7fb5400a9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702fc47f5f5dc6ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.552+0000 7fb54cc00700 1 -- 172.21.15.50:0/1418702373 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400094d0 con 0x55a4821789a0 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.552+0000 7fb54cc00700 1 -- 172.21.15.50:0/1418702373 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5400adc30 con 0x55a4821789a0 2024-05-10T21:07:18.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.552+0000 7fb54cc00700 1 -- 172.21.15.50:0/1418702373 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400a9d40 con 0x55a4821789a0 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/1418702373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 msgr2=0x55a482180c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9c0 tx=0x7fb5400a9780 comp rx=0 tx=0).stop 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/1418702373 shutdown_connections 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a482178460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/1418702373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a482180c20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/1418702373 >> 172.21.15.50:0/1418702373 conn(0x55a48203ff20 msgr2=0x55a48217d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/1418702373 shutdown_connections 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.553+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/1418702373 wait complete. 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 -- start start 2024-05-10T21:07:18.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 Processor -- start 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4821936e0 con 0x55a4821789a0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a482193010 con 0x55a481ffadf0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a482193db0 con 0x55a48217e7e0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.554+0000 7fb56d4d1b80 10 client.0 _ll_register_callbacks cb 0x55a482041c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb557e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59662/0 (socket says 172.21.15.50:59662) 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 -- 172.21.15.50:0/2824893139 learned_addr learned my addr 172.21.15.50:0/2824893139 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb557e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40122/0 (socket says 172.21.15.50:40122) 2024-05-10T21:07:18.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb55ca00700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 -- 172.21.15.50:0/2824893139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 msgr2=0x55a48218db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 -- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 msgr2=0x55a48219d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb54f400700 1 -- 172.21.15.50:0/2824893139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48218e310 con 0x55a4821789a0 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.555+0000 7fb557e00700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb54f400700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c5b0 tx=0x7fb540003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc06137cb5d8b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:18.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb546000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400c5070 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb5400cc0f0 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5400adc30 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5400b26e0 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb56d4d1b80 10 client.14805 fetch_fsmap learned FSMap version 24 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4821fd860 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.556+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5400adc30 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb5400d9020 con 0x55a4821789a0 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 10 client.14805 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 10 client.14805 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 20 client.14805 populate_metadata read hostname 'smithi050' 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821ff510 0x55a482200980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 4 client.14805 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:07:18.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "status"}) -- 0x55a4820a83b0 con 0x55a4821ff510 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.557+0000 7fb557e00700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821ff510 0x55a482200980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb557e00700 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821ff510 0x55a482200980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef7a812ae9abd19 server_cookie=45fdafc48c47aab1 in_seq=0 out_seq=0 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb546000700 10 client.14805 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb546000700 1 -- 172.21.15.50:0/2824893139 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a4820a83b0 con 0x55a4821ff510 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb546000700 10 client.14805 handle_command_reply: tid=0 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 client.14805 shutdown 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 msgr2=0x55a48218d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c5b0 tx=0x7fb540003700 comp rx=0 tx=0).stop 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 shutdown_connections 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a4821ff510 0x55a482200980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a481ffadf0 0x55a48219d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a48217e7e0 0x55a48218db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 --2- 172.21.15.50:0/2824893139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4821789a0 0x55a48218d5d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.558+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 >> 172.21.15.50:0/2824893139 conn(0x55a48203ff20 msgr2=0x55a482191870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb546000700 0 client.14805 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 shutdown_connections 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb56d4d1b80 1 -- 172.21.15.50:0/2824893139 wait complete. 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb56d4d1b80 20 client.14805 trim_cache size 0 max 16384 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24539, 2024-05-10T21:07:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 171.61533180500001, 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 194.19607746099999 2024-05-10T21:07:18.564 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 msgr2=0x7fb53c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.559+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 0x7fb53c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb550009fd0 tx=0x7fb55000b040 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 msgr2=0x7fb5581a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 0x7fb5581a19a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9f0 tx=0x7fb54000ed00 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 shutdown_connections 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb53c0b2900 0x7fb53c0b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb558115370 0x7fb5581a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb558075b60 0x7fb5581a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 --2- 172.21.15.50:0/265798149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5580764b0 0x7fb5581a19a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 >> 172.21.15.50:0/265798149 conn(0x7fb558100040 msgr2=0x7fb5581006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 shutdown_connections 2024-05-10T21:07:18.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:18.560+0000 7fb55ea00700 1 -- 172.21.15.50:0/265798149 wait complete. 2024-05-10T21:07:18.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:18 smithi050 ceph-mon[29546]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:19.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:18 smithi092 ceph-mon[34415]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:19.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:18 smithi142 ceph-mon[34846]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:19.423 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T21:07:19.423 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T21:07:19.434 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:07:19.435 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T21:07:19.923 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:20.966 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:20 smithi050 ceph-mon[29546]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:21.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:20 smithi092 ceph-mon[34415]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:21.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:20 smithi142 ceph-mon[34846]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 -- 172.21.15.50:0/3891752693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010a790 msgr2=0x7fc0a010aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/3891752693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010a790 0x7fc0a010aba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc090009990 tx=0x7fc090009ca0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 -- 172.21.15.50:0/3891752693 shutdown_connections 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/3891752693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 0x7fc0a0112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/3891752693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010a790 0x7fc0a010aba0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/3891752693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010b8c0 0x7fc0a010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 -- 172.21.15.50:0/3891752693 >> 172.21.15.50:0/3891752693 conn(0x7fc0a0079840 msgr2=0x7fc0a01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.195+0000 7fc0a4e00700 1 -- 172.21.15.50:0/3891752693 shutdown_connections 2024-05-10T21:07:22.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.196+0000 7fc0a4e00700 1 -- 172.21.15.50:0/3891752693 wait complete. 2024-05-10T21:07:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 Processor -- start 2024-05-10T21:07:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 -- start start 2024-05-10T21:07:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010b8c0 0x7fc0a01a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:22.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 0x7fc0a01a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01161d0 con 0x7fc0a010c260 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0116050 con 0x7fc0a010a790 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.197+0000 7fc0a4e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0116350 con 0x7fc0a010b8c0 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40132/0 (socket says 172.21.15.50:40132) 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 -- 172.21.15.50:0/2643202909 learned_addr learned my addr 172.21.15.50:0/2643202909 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09fe00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 0x7fc0a01a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09ea00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010b8c0 0x7fc0a01a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 -- 172.21.15.50:0/2643202909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010b8c0 msgr2=0x7fc0a01a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010b8c0 0x7fc0a01a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 -- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 msgr2=0x7fc0a01a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:22.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 0x7fc0a01a81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 -- 172.21.15.50:0/2643202909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a01a88a0 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.198+0000 7fc09f400700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc090006010 tx=0x7fc0900047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.199+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090026070 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.199+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc090021410 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.199+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a01a8b30 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.199+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090017650 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.199+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01a8f80 con 0x7fc0a010a790 2024-05-10T21:07:22.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.200+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a00512c0 con 0x7fc0a010a790 2024-05-10T21:07:22.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.203+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc090029070 con 0x7fc0a010a790 2024-05-10T21:07:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.204+0000 7fc097e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 0x7fc0840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:22.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.204+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc0900e3790 con 0x7fc0a010a790 2024-05-10T21:07:22.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.204+0000 7fc09ea00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 0x7fc0840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:22.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.205+0000 7fc09ea00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 0x7fc0840b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009c60 tx=0x7fc08c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:22.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.206+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc090030020 con 0x7fc0a010a790 2024-05-10T21:07:22.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.460+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc0a0063c00 con 0x7fc0840b2880 2024-05-10T21:07:22.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.464+0000 7fc097e00700 1 -- 172.21.15.50:0/2643202909 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc0a0063c00 con 0x7fc0840b2880 2024-05-10T21:07:22.465 INFO:teuthology.orchestra.run.smithi050.stdout:Backend: cephadm 2024-05-10T21:07:22.465 INFO:teuthology.orchestra.run.smithi050.stdout:Available: Yes 2024-05-10T21:07:22.465 INFO:teuthology.orchestra.run.smithi050.stdout:Paused: No 2024-05-10T21:07:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 msgr2=0x7fc0840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 0x7fc0840b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009c60 tx=0x7fc08c009260 comp rx=0 tx=0).stop 2024-05-10T21:07:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 msgr2=0x7fc0a01a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:22.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc090006010 tx=0x7fc0900047b0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 shutdown_connections 2024-05-10T21:07:22.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc0840b2880 0x7fc0840b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a010a790 0x7fc0a01a0d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0a010c260 0x7fc0a01a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 --2- 172.21.15.50:0/2643202909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a010b8c0 0x7fc0a01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.467+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 >> 172.21.15.50:0/2643202909 conn(0x7fc0a0079840 msgr2=0x7fc0a010eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.468+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 shutdown_connections 2024-05-10T21:07:22.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:22.468+0000 7fc0a4e00700 1 -- 172.21.15.50:0/2643202909 wait complete. 2024-05-10T21:07:23.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:22 smithi092 ceph-mon[34415]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:23.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:22 smithi142 ceph-mon[34846]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:22 smithi050 ceph-mon[29546]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:23.291 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T21:07:23.765 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:24.017 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:23 smithi050 ceph-mon[29546]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:24.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:23 smithi092 ceph-mon[34415]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:24.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:23 smithi142 ceph-mon[34846]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:25.030 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:24 smithi050 ceph-mon[29546]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:25.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:24 smithi092 ceph-mon[34415]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:25.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:24 smithi142 ceph-mon[34846]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 -- 172.21.15.50:0/2258833795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f00760f0 msgr2=0x7f82f0076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 --2- 172.21.15.50:0/2258833795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f00760f0 0x7f82f0076580 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82d80099c0 tx=0x7f82d8009cd0 comp rx=0 tx=0).stop 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 -- 172.21.15.50:0/2258833795 shutdown_connections 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 --2- 172.21.15.50:0/2258833795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00776e0 0x7f82f0075bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 --2- 172.21.15.50:0/2258833795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f0110990 0x7f82f0112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 --2- 172.21.15.50:0/2258833795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f00760f0 0x7f82f0076580 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 -- 172.21.15.50:0/2258833795 >> 172.21.15.50:0/2258833795 conn(0x7f82f00ff9c0 msgr2=0x7f82f0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:25.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.995+0000 7f82f6000700 1 -- 172.21.15.50:0/2258833795 shutdown_connections 2024-05-10T21:07:25.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.996+0000 7f82f6000700 1 -- 172.21.15.50:0/2258833795 wait complete. 2024-05-10T21:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 Processor -- start 2024-05-10T21:07:26.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 -- start start 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 0x7f82f01a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f00776e0 0x7f82f01a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0116400 con 0x7f82f00776e0 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0116280 con 0x7f82f00760f0 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.997+0000 7f82f6000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0116580 con 0x7f82f0110990 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82e6a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f00776e0 0x7f82f01a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46024/0 (socket says 172.21.15.50:46024) 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 -- 172.21.15.50:0/261831519 learned_addr learned my addr 172.21.15.50:0/261831519 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:26.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82ef400700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 0x7f82f01a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 -- 172.21.15.50:0/261831519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 msgr2=0x7f82f01a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 0x7f82f01a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 -- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f00776e0 msgr2=0x7f82f01a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f00776e0 0x7f82f01a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82efe00700 1 -- 172.21.15.50:0/261831519 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f01a8a90 con 0x7f82f0110990 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.998+0000 7f82ef400700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 0x7f82f01a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82efe00700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82e800ef30 tx=0x7f82e800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80092e0 con 0x7f82f0110990 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e8004bb0 con 0x7f82f0110990 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004020 con 0x7f82f0110990 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f01a8d20 con 0x7f82f0110990 2024-05-10T21:07:26.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:25.999+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01a91b0 con 0x7f82f0110990 2024-05-10T21:07:26.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.000+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f00512c0 con 0x7f82f0110990 2024-05-10T21:07:26.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.003+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f82e801b030 con 0x7f82f0110990 2024-05-10T21:07:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.004+0000 7f82ecc00700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 0x7f82d00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.004+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f82e80e2690 con 0x7f82f0110990 2024-05-10T21:07:26.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.005+0000 7f82ef400700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 0x7f82d00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:26.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.005+0000 7f82ef400700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 0x7f82d00b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82f00773e0 tx=0x7f82e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:26.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.006+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82e8022020 con 0x7f82f0110990 2024-05-10T21:07:26.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.256+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f82f0063c00 con 0x7f82d00b2900 2024-05-10T21:07:26.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.272+0000 7f82ecc00700 1 -- 172.21.15.50:0/261831519 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f82f0063c00 con 0x7f82d00b2900 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:mds.a smithi050 running (3m) 44s ago 3m 86.0M - 19.0.0-3590-g7b710596 d1bf1c877c43 28049d57f73f 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:mds.b smithi092 running (3m) 64s ago 3m 93.2M - 19.0.0-3590-g7b710596 d1bf1c877c43 f3862f24d6c0 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:mds.c smithi142 running (3m) 64s ago 3m 86.0M - 19.0.0-3590-g7b710596 d1bf1c877c43 23ac4e1a9133 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:mds.d smithi050 running (3m) 44s ago 3m 80.8M - 19.0.0-3590-g7b710596 d1bf1c877c43 16828d83ab51 2024-05-10T21:07:26.274 INFO:teuthology.orchestra.run.smithi050.stdout:mds.e smithi092 running (3m) 64s ago 3m 86.5M - 19.0.0-3590-g7b710596 d1bf1c877c43 fb2bdcff094c 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.f smithi142 running (3m) 64s ago 3m 81.1M - 19.0.0-3590-g7b710596 d1bf1c877c43 d10ac3efcd79 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.g smithi050 running (3m) 44s ago 3m 82.7M - 19.0.0-3590-g7b710596 d1bf1c877c43 68383664576f 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.h smithi092 running (3m) 64s ago 3m 90.0M - 19.0.0-3590-g7b710596 d1bf1c877c43 6faac74d2d73 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.i smithi142 running (3m) 64s ago 3m 89.4M - 19.0.0-3590-g7b710596 d1bf1c877c43 9afff81020a4 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.j smithi050 running (3m) 44s ago 3m 80.7M - 19.0.0-3590-g7b710596 d1bf1c877c43 dfc2796d73f3 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.k smithi092 running (3m) 64s ago 3m 83.5M - 19.0.0-3590-g7b710596 d1bf1c877c43 d294a4f8d7d7 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mds.l smithi142 running (4m) 64s ago 4m 81.5M - 19.0.0-3590-g7b710596 d1bf1c877c43 f21d04795d09 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.x smithi050 *:9283,8765 running (12m) 44s ago 12m 598M - 19.0.0-3590-g7b710596 d1bf1c877c43 806c29320aa8 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.y smithi092 *:8443,8765 running (10m) 64s ago 10m 538M - 19.0.0-3590-g7b710596 d1bf1c877c43 8eef36b1a820 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.z smithi142 *:8443,8765 running (10m) 64s ago 10m 537M - 19.0.0-3590-g7b710596 d1bf1c877c43 f24f09d898e3 2024-05-10T21:07:26.275 INFO:teuthology.orchestra.run.smithi050.stdout:mon.a smithi050 running (12m) 44s ago 12m 117M 2048M 19.0.0-3590-g7b710596 d1bf1c877c43 1ba2cba890c2 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:mon.b smithi092 running (10m) 64s ago 10m 110M 2048M 19.0.0-3590-g7b710596 d1bf1c877c43 a3aacc932add 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:mon.c smithi142 running (10m) 64s ago 10m 112M 2048M 19.0.0-3590-g7b710596 d1bf1c877c43 cdfeffead09c 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 smithi050 running (9m) 44s ago 9m 115M 4273M 19.0.0-3590-g7b710596 d1bf1c877c43 12c163af1298 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 smithi092 running (9m) 64s ago 9m 124M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 6d4e9ac5f5b9 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 smithi142 running (8m) 64s ago 8m 119M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 abe5f581c3dc 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 smithi050 running (8m) 44s ago 8m 114M 4273M 19.0.0-3590-g7b710596 d1bf1c877c43 30a31a843e5e 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 smithi092 running (7m) 64s ago 7m 124M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 014aba315de4 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 smithi142 running (7m) 64s ago 7m 112M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 e24b7a4ff3f3 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 smithi050 running (6m) 44s ago 6m 113M 4273M 19.0.0-3590-g7b710596 d1bf1c877c43 46152bead5ff 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 smithi092 running (6m) 64s ago 6m 122M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 b565d95be35d 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 smithi142 running (5m) 64s ago 5m 126M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 c55d5ab051cf 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 smithi050 running (5m) 44s ago 5m 113M 4273M 19.0.0-3590-g7b710596 d1bf1c877c43 f8f98545bf4c 2024-05-10T21:07:26.276 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 smithi092 running (4m) 64s ago 4m 125M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 7510d4927af1 2024-05-10T21:07:26.277 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 smithi142 running (4m) 64s ago 4m 120M 4272M 19.0.0-3590-g7b710596 d1bf1c877c43 cb90a9a931c6 2024-05-10T21:07:26.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 msgr2=0x7f82d00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:26.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 0x7f82d00b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82f00773e0 tx=0x7f82e0009260 comp rx=0 tx=0).stop 2024-05-10T21:07:26.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 msgr2=0x7f82f01a8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82e800ef30 tx=0x7f82e800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 shutdown_connections 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f82d00b2900 0x7f82d00b4e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82f00760f0 0x7f82f01a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f0110990 0x7f82f01a8410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 --2- 172.21.15.50:0/261831519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f82f00776e0 0x7f82f01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.275+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 >> 172.21.15.50:0/261831519 conn(0x7f82f00ff9c0 msgr2=0x7f82f0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.276+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 shutdown_connections 2024-05-10T21:07:26.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:26.276+0000 7f82f6000700 1 -- 172.21.15.50:0/261831519 wait complete. 2024-05-10T21:07:26.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:26 smithi050 ceph-mon[29546]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:27.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T21:07:27.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:26 smithi092 ceph-mon[34415]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:27.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:26 smithi142 ceph-mon[34846]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:27.553 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:28.044 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:27 smithi050 ceph-mon[29546]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:28.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:27 smithi092 ceph-mon[34415]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:28.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:27 smithi142 ceph-mon[34846]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:28.864 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:28 smithi050 ceph-mon[29546]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:29.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:28 smithi092 ceph-mon[34415]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:28 smithi142 ceph-mon[34846]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.790+0000 7f489d600700 1 -- 172.21.15.50:0/270317283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4898107490 msgr2=0x7f4898109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.790+0000 7f489d600700 1 --2- 172.21.15.50:0/270317283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4898107490 0x7f4898109970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f48800098f0 tx=0x7f4880009c00 comp rx=0 tx=0).stop 2024-05-10T21:07:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.790+0000 7f489d600700 1 -- 172.21.15.50:0/270317283 shutdown_connections 2024-05-10T21:07:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.790+0000 7f489d600700 1 --2- 172.21.15.50:0/270317283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f489810a000 0x7f489810c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:29.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.790+0000 7f489d600700 1 --2- 172.21.15.50:0/270317283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48980766b0 0x7f4898106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.791+0000 7f489d600700 1 --2- 172.21.15.50:0/270317283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4898107490 0x7f4898109970 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.791+0000 7f489d600700 1 -- 172.21.15.50:0/270317283 >> 172.21.15.50:0/270317283 conn(0x7f48980fe890 msgr2=0x7f4898100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.791+0000 7f489d600700 1 -- 172.21.15.50:0/270317283 shutdown_connections 2024-05-10T21:07:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.791+0000 7f489d600700 1 -- 172.21.15.50:0/270317283 wait complete. 2024-05-10T21:07:29.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.792+0000 7f489d600700 1 Processor -- start 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.792+0000 7f489d600700 1 -- start start 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4898107490 0x7f489819d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f489810fde0 con 0x7f489810a000 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f489810fc60 con 0x7f48980766b0 2024-05-10T21:07:29.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f489d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f489810ff60 con 0x7f4898107490 2024-05-10T21:07:29.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4897400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:29.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4897400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57698/0 (socket says 172.21.15.50:57698) 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46234/0 (socket says 172.21.15.50:46234) 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f488e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4898107490 0x7f489819d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4897400700 1 -- 172.21.15.50:0/2014499377 learned_addr learned my addr 172.21.15.50:0/2014499377 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 -- 172.21.15.50:0/2014499377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4898107490 msgr2=0x7f489819d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4898107490 0x7f489819d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 -- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 msgr2=0x7f48981a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4896a00700 1 -- 172.21.15.50:0/2014499377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48981a46e0 con 0x7f48980766b0 2024-05-10T21:07:29.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.793+0000 7f4897400700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f4896a00700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f488800ef10 tx=0x7f488800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800c9c0 con 0x7f48980766b0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4888019070 con 0x7f48980766b0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888014480 con 0x7f48980766b0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48981a4970 con 0x7f48980766b0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.794+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981a4f00 con 0x7f48980766b0 2024-05-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.795+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48980512c0 con 0x7f48980766b0 2024-05-10T21:07:29.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.798+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4888012050 con 0x7f48980766b0 2024-05-10T21:07:29.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.800+0000 7f488fe00700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:29.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.800+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f48880e2730 con 0x7f48980766b0 2024-05-10T21:07:29.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.800+0000 7f488e000700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:29.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.801+0000 7f488e000700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 0x7f487c0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f489819dc30 tx=0x7f4880019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:29.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:29.802+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4888010040 con 0x7f48980766b0 2024-05-10T21:07:30.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.049+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f48981a51f0 con 0x7f487c0b2900 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.069+0000 7f488fe00700 1 -- 172.21.15.50:0/2014499377 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f48981a51f0 con 0x7f487c0b2900 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stdout:mds.all 12/12 68s ago 4m smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stdout:mgr 3/3 68s ago 10m smithi050=x;smithi092=y;smithi142=z;count:3 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stdout:mon 3/3 68s ago 10m smithi050:172.21.15.50=a;smithi092:172.21.15.92=b;smithi142:172.21.15.142=c;count:3 2024-05-10T21:07:30.070 INFO:teuthology.orchestra.run.smithi050.stdout:osd 12 68s ago - 2024-05-10T21:07:30.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 msgr2=0x7f487c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:30.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 0x7f487c0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f489819dc30 tx=0x7f4880019040 comp rx=0 tx=0).stop 2024-05-10T21:07:30.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 msgr2=0x7f489819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:30.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f488800ef10 tx=0x7f488800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 shutdown_connections 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f487c0b2900 0x7f487c0b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48980766b0 0x7f489819cb80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489810a000 0x7f48981a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 --2- 172.21.15.50:0/2014499377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4898107490 0x7f489819d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 >> 172.21.15.50:0/2014499377 conn(0x7f48980fe890 msgr2=0x7f4898100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 shutdown_connections 2024-05-10T21:07:30.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:30.072+0000 7f489d600700 1 -- 172.21.15.50:0/2014499377 wait complete. 2024-05-10T21:07:30.857 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:30 smithi050 ceph-mon[29546]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:30.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T21:07:31.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:30 smithi092 ceph-mon[34415]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:31.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:30 smithi142 ceph-mon[34846]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:31.406 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:31.894 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:31 smithi050 ceph-mon[29546]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:32.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:31 smithi092 ceph-mon[34415]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:32.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:31 smithi142 ceph-mon[34846]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:33.056 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:32 smithi050 ceph-mon[29546]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:33.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:32 smithi092 ceph-mon[34415]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:33.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:32 smithi142 ceph-mon[34846]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 -- 172.21.15.50:0/1203853419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 msgr2=0x7fc9e01058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 --2- 172.21.15.50:0/1203853419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e01058b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009990 tx=0x7fc9c8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 -- 172.21.15.50:0/1203853419 shutdown_connections 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 --2- 172.21.15.50:0/1203853419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0105e40 0x7fc9e010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 --2- 172.21.15.50:0/1203853419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0104370 0x7fc9e0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 --2- 172.21.15.50:0/1203853419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e01058b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 -- 172.21.15.50:0/1203853419 >> 172.21.15.50:0/1203853419 conn(0x7fc9e00ff9d0 msgr2=0x7fc9e0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 -- 172.21.15.50:0/1203853419 shutdown_connections 2024-05-10T21:07:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.603+0000 7fc9e6600700 1 -- 172.21.15.50:0/1203853419 wait complete. 2024-05-10T21:07:33.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.604+0000 7fc9e6600700 1 Processor -- start 2024-05-10T21:07:33.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 -- start start 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 0x7fc9e019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 0x7fc9e01a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010fdd0 con 0x7fc9e0105e40 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010fc50 con 0x7fc9e01054a0 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9e6600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010ff50 con 0x7fc9e0104370 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9dfe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 0x7fc9e01a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 0x7fc9e019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 0x7fc9e019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46078/0 (socket says 172.21.15.50:46078) 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9df400700 1 -- 172.21.15.50:0/2111049686 learned_addr learned my addr 172.21.15.50:0/2111049686 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:33.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.605+0000 7fc9d6a00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:33.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9dfe00700 1 -- 172.21.15.50:0/2111049686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 msgr2=0x7fc9e019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:33.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9dfe00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 0x7fc9e019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9dfe00700 1 -- 172.21.15.50:0/2111049686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 msgr2=0x7fc9e019d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9dfe00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e019d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9dfe00700 1 -- 172.21.15.50:0/2111049686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e01a46c0 con 0x7fc9e0105e40 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.606+0000 7fc9d6a00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9dfe00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 0x7fc9e01a4040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800efb0 tx=0x7fc9d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8003a00 con 0x7fc9e0105e40 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e01a49b0 con 0x7fc9e0105e40 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d800ce50 con 0x7fc9e0105e40 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80047c0 con 0x7fc9e0105e40 2024-05-10T21:07:33.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.607+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a4ea0 con 0x7fc9e0105e40 2024-05-10T21:07:33.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.609+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00512c0 con 0x7fc9e0105e40 2024-05-10T21:07:33.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.611+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc9d8012050 con 0x7fc9e0105e40 2024-05-10T21:07:33.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.613+0000 7fc9dcc00700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 0x7fc9c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:33.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.614+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc9d80e1af0 con 0x7fc9e0105e40 2024-05-10T21:07:33.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.614+0000 7fc9df400700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 0x7fc9c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:33.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.614+0000 7fc9df400700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 0x7fc9c40b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009cc0 tx=0x7fc9d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:33.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.623+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9d8022060 con 0x7fc9e0105e40 2024-05-10T21:07:33.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.871+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc9e0078220 con 0x7fc9c40b2880 2024-05-10T21:07:33.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.874+0000 7fc9dcc00700 1 -- 172.21.15.50:0/2111049686 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc9e0078220 con 0x7fc9c40b2880 2024-05-10T21:07:33.876 INFO:teuthology.orchestra.run.smithi050.stdout:HOST ADDR LABELS STATUS 2024-05-10T21:07:33.876 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 172.21.15.50 2024-05-10T21:07:33.876 INFO:teuthology.orchestra.run.smithi050.stdout:smithi092 172.21.15.92 2024-05-10T21:07:33.876 INFO:teuthology.orchestra.run.smithi050.stdout:smithi142 172.21.15.142 2024-05-10T21:07:33.876 INFO:teuthology.orchestra.run.smithi050.stdout:3 hosts in cluster 2024-05-10T21:07:33.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 msgr2=0x7fc9c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:33.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 0x7fc9c40b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009cc0 tx=0x7fc9d0009260 comp rx=0 tx=0).stop 2024-05-10T21:07:33.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 msgr2=0x7fc9e01a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:33.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 0x7fc9e01a4040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800efb0 tx=0x7fc9d800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 shutdown_connections 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc9c40b2880 0x7fc9c40b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9e0105e40 0x7fc9e01a4040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9e0104370 0x7fc9e019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 --2- 172.21.15.50:0/2111049686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e01054a0 0x7fc9e019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 >> 172.21.15.50:0/2111049686 conn(0x7fc9e00ff9d0 msgr2=0x7fc9e0076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.878+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 shutdown_connections 2024-05-10T21:07:33.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:33.879+0000 7fc9e6600700 1 -- 172.21.15.50:0/2111049686 wait complete. 2024-05-10T21:07:34.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T21:07:34.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:34 smithi050 ceph-mon[29546]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:34.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:34 smithi050 ceph-mon[29546]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:35.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:34 smithi092 ceph-mon[34415]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:35.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:34 smithi092 ceph-mon[34415]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:35.121 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:35.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:34 smithi142 ceph-mon[34846]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T21:07:35.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:34 smithi142 ceph-mon[34846]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:36.855 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:36 smithi050 ceph-mon[29546]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:37.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:36 smithi092 ceph-mon[34415]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:37.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:36 smithi142 ceph-mon[34846]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 -- 172.21.15.50:0/4169999747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a807ace0 msgr2=0x7f53a807b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 --2- 172.21.15.50:0/4169999747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a807ace0 0x7f53a807b5d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009990 tx=0x7f53a0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 -- 172.21.15.50:0/4169999747 shutdown_connections 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 --2- 172.21.15.50:0/4169999747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a807ace0 0x7f53a807b5d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 --2- 172.21.15.50:0/4169999747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a8079210 0x7f53a8079620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 --2- 172.21.15.50:0/4169999747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807a390 0x7f53a807a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 -- 172.21.15.50:0/4169999747 >> 172.21.15.50:0/4169999747 conn(0x7f53a8101bd0 msgr2=0x7f53a8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 -- 172.21.15.50:0/4169999747 shutdown_connections 2024-05-10T21:07:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.432+0000 7f53afc00700 1 -- 172.21.15.50:0/4169999747 wait complete. 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.433+0000 7f53afc00700 1 Processor -- start 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 -- start start 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a807a390 0x7f53a81a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807ace0 0x7f53a81a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8116470 con 0x7f53a8079210 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81162f0 con 0x7f53a807ace0 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53afc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81165f0 con 0x7f53a807a390 2024-05-10T21:07:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.434+0000 7f53ad200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49274/0 (socket says 172.21.15.50:49274) 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 -- 172.21.15.50:0/730787542 learned_addr learned my addr 172.21.15.50:0/730787542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53a7e00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a807a390 0x7f53a81a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53adc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807ace0 0x7f53a81a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 -- 172.21.15.50:0/730787542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a807a390 msgr2=0x7f53a81a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a807a390 0x7f53a81a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 -- 172.21.15.50:0/730787542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807ace0 msgr2=0x7f53a81a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807ace0 0x7f53a81a83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.435+0000 7f53ad200700 1 -- 172.21.15.50:0/730787542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a81a8a30 con 0x7f53a8079210 2024-05-10T21:07:37.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53ad200700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f539c00efc0 tx=0x7f539c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:37.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c00ca70 con 0x7f53a8079210 2024-05-10T21:07:37.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f539c019070 con 0x7f53a8079210 2024-05-10T21:07:37.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c0146e0 con 0x7f53a8079210 2024-05-10T21:07:37.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a81a8cc0 con 0x7f53a8079210 2024-05-10T21:07:37.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.436+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a91a0 con 0x7f53a8079210 2024-05-10T21:07:37.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.437+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a80512c0 con 0x7f53a8079210 2024-05-10T21:07:37.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.440+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f539c012050 con 0x7f53a8079210 2024-05-10T21:07:37.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.441+0000 7f53a5600700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 0x7f53900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:37.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.441+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f539c0e28c0 con 0x7f53a8079210 2024-05-10T21:07:37.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.442+0000 7f53a7e00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 0x7f53900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:37.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.442+0000 7f53a7e00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 0x7f53900b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53a81a1f80 tx=0x7f5394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:37.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.444+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f539c010040 con 0x7f53a8079210 2024-05-10T21:07:37.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.688+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f53a8063c00 con 0x7f53900b2900 2024-05-10T21:07:37.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.692+0000 7f53a5600700 1 -- 172.21.15.50:0/730787542 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f53a8063c00 con 0x7f53900b2900 2024-05-10T21:07:37.692 INFO:teuthology.orchestra.run.smithi050.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T21:07:37.692 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-05-10T21:07:37.692 INFO:teuthology.orchestra.run.smithi050.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-10T21:07:37.692 INFO:teuthology.orchestra.run.smithi050.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-10T21:07:37.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 msgr2=0x7f53900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 0x7f53900b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53a81a1f80 tx=0x7f5394009260 comp rx=0 tx=0).stop 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 msgr2=0x7f53a81a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f539c00efc0 tx=0x7f539c00c3a0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 shutdown_connections 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53900b2900 0x7f53900b4e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53a807ace0 0x7f53a81a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8079210 0x7f53a81a0ed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 --2- 172.21.15.50:0/730787542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53a807a390 0x7f53a81a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 >> 172.21.15.50:0/730787542 conn(0x7f53a8101bd0 msgr2=0x7f53a8110820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 shutdown_connections 2024-05-10T21:07:37.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:37.696+0000 7f53afc00700 1 -- 172.21.15.50:0/730787542 wait complete. 2024-05-10T21:07:38.521 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T21:07:38.531 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:07:38.531 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T21:07:38.987 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:39.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:38 smithi092 ceph-mon[34415]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:39.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:38 smithi092 ceph-mon[34415]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:39.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:38 smithi142 ceph-mon[34846]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:39.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:38 smithi142 ceph-mon[34846]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:39.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:38 smithi050 ceph-mon[29546]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:39.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:38 smithi050 ceph-mon[29546]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T21:07:41.042 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:40 smithi050 ceph-mon[29546]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:41.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:40 smithi092 ceph-mon[34415]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:41.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:40 smithi142 ceph-mon[34846]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:41.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 -- 172.21.15.50:0/3639132643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 msgr2=0x7fce8810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 --2- 172.21.15.50:0/3639132643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce8810b5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce80009f90 tx=0x7fce8000b330 comp rx=0 tx=0).stop 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 -- 172.21.15.50:0/3639132643 shutdown_connections 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 --2- 172.21.15.50:0/3639132643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88103240 0x7fce88103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 --2- 172.21.15.50:0/3639132643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce8810b5c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 --2- 172.21.15.50:0/3639132643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88104370 0x7fce88104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 -- 172.21.15.50:0/3639132643 >> 172.21.15.50:0/3639132643 conn(0x7fce880fe880 msgr2=0x7fce88100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 -- 172.21.15.50:0/3639132643 shutdown_connections 2024-05-10T21:07:41.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.387+0000 7fce90600700 1 -- 172.21.15.50:0/3639132643 wait complete. 2024-05-10T21:07:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.388+0000 7fce90600700 1 Processor -- start 2024-05-10T21:07:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.388+0000 7fce90600700 1 -- start start 2024-05-10T21:07:41.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 0x7fce8819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce881a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810ed00 con 0x7fce88103240 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810eb80 con 0x7fce88104370 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce90600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810ee80 con 0x7fce88104d10 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce8dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce8dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49292/0 (socket says 172.21.15.50:49292) 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.389+0000 7fce8dc00700 1 -- 172.21.15.50:0/3582914576 learned_addr learned my addr 172.21.15.50:0/3582914576 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8e600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce881a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:41.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8d200700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 0x7fce8819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 -- 172.21.15.50:0/3582914576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 msgr2=0x7fce881a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce881a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 -- 172.21.15.50:0/3582914576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 msgr2=0x7fce8819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 0x7fce8819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 -- 172.21.15.50:0/3582914576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce881a4640 con 0x7fce88103240 2024-05-10T21:07:41.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8d200700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 0x7fce8819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.390+0000 7fce8dc00700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e9c0 tx=0x7fce7c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.391+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c004220 con 0x7fce88103240 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.391+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce881a4930 con 0x7fce88103240 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.391+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce7c0043c0 con 0x7fce88103240 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.391+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c003bb0 con 0x7fce88103240 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.391+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a4ec0 con 0x7fce88103240 2024-05-10T21:07:41.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.392+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce880512c0 con 0x7fce88103240 2024-05-10T21:07:41.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.395+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce7c004870 con 0x7fce88103240 2024-05-10T21:07:41.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.396+0000 7fce86000700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 0x7fce700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:41.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.397+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fce7c0e24b0 con 0x7fce88103240 2024-05-10T21:07:41.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.397+0000 7fce8d200700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 0x7fce700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:41.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.398+0000 7fce8d200700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 0x7fce700b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce8819db90 tx=0x7fce74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:41.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.404+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce7c0aa290 con 0x7fce88103240 2024-05-10T21:07:41.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.686+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fce881a5150 con 0x7fce88103240 2024-05-10T21:07:41.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.686+0000 7fce86000700 1 -- 172.21.15.50:0/3582914576 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3038 (secure 0 0 0) 0x7fce7c0a9b90 con 0x7fce88103240 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:e24 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:legacy client fscid: 1 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:Filesystem 'cephfs' (1) 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:fs_name cephfs 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 24 2024-05-10T21:07:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-05-10T21:04:26.742871+0000 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-05-10T21:04:50.607977+0000 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:tableserver 0 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:root 0 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:session_timeout 300 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:session_autoclose 300 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:max_file_size 1099511627776 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:max_xattr_size 65536 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:required_client_features {} 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure 0 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure_osd_epoch 0 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:max_mds 3 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:in 0,1,2 2024-05-10T21:07:41.689 INFO:teuthology.orchestra.run.smithi050.stdout:up {0=24539,1=24520,2=24502} 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:failed 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:damaged 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:stopped 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:data_pools [3] 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:metadata_pool 2 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:inline_data disabled 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:balancer 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:bal_rank_mask -1 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:standby_count_wanted 0 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:qdb_cluster leader: 24539 members: 24539,24502,24520 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b{0:24539} state up:active seq 7 addr [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.h{0:24500} state up:standby-replay seq 1 addr [v2:172.21.15.92:6836/1752993045,v1:172.21.15.92:6837/1752993045] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a{1:24520} state up:active seq 12 addr [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c{1:24490} state up:standby-replay seq 1 addr [v2:172.21.15.142:6836/924361476,v1:172.21.15.142:6837/924361476] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.i{2:24502} state up:active seq 14 addr [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.e{2:24482} state up:standby-replay seq 1 addr [v2:172.21.15.92:6834/1464775005,v1:172.21.15.92:6835/1464775005] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:Standby daemons: 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.j{-1:14367} state up:standby seq 1 addr [v2:172.21.15.50:6834/1323665657,v1:172.21.15.50:6835/1323665657] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.g{-1:14397} state up:standby seq 1 addr [v2:172.21.15.50:6838/82705098,v1:172.21.15.50:6839/82705098] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.142:6832/2643414360,v1:172.21.15.142:6833/2643414360] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.k{-1:24452} state up:standby seq 1 addr [v2:172.21.15.92:6832/194068448,v1:172.21.15.92:6833/194068448] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d{-1:24460} state up:standby seq 1 addr [v2:172.21.15.50:6836/1369162577,v1:172.21.15.50:6837/1369162577] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.691 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.f{-1:24478} state up:standby seq 1 addr [v2:172.21.15.142:6834/86448197,v1:172.21.15.142:6835/86448197] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 msgr2=0x7fce700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 0x7fce700b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce8819db90 tx=0x7fce74009260 comp rx=0 tx=0).stop 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 msgr2=0x7fce8819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e9c0 tx=0x7fce7c00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 shutdown_connections 2024-05-10T21:07:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fce700b2880 0x7fce700b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce88104d10 0x7fce881a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce88103240 0x7fce8819cae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 --2- 172.21.15.50:0/3582914576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce88104370 0x7fce8819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 >> 172.21.15.50:0/3582914576 conn(0x7fce880fe880 msgr2=0x7fce88100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.689+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 shutdown_connections 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:41.690+0000 7fce90600700 1 -- 172.21.15.50:0/3582914576 wait complete. 2024-05-10T21:07:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:07:42.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:41 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3582914576' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T21:07:42.121 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:41 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3582914576' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T21:07:42.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:41 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3582914576' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T21:07:42.554 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T21:07:43.011 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:43.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:42 smithi092 ceph-mon[34415]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:43.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:42 smithi142 ceph-mon[34846]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:43.157 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:42 smithi050 ceph-mon[29546]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:44.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:07:44.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:43 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:07:44.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:43 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:07:44.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:43 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:44.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:44 smithi050 ceph-mon[29546]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:44.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:07:44.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:07:44.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:45.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:44 smithi092 ceph-mon[34415]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:45.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:07:45.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:07:45.120 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:45.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:44 smithi142 ceph-mon[34846]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:45.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:07:45.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:07:45.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:07:45.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 -- 172.21.15.50:0/1854061725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec105470 msgr2=0x7f30ec105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 --2- 172.21.15.50:0/1854061725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec105470 0x7f30ec105880 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b0a0 tx=0x7f30d400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 -- 172.21.15.50:0/1854061725 shutdown_connections 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 --2- 172.21.15.50:0/1854061725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec104340 0x7f30ec104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 --2- 172.21.15.50:0/1854061725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105e10 0x7f30ec10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 --2- 172.21.15.50:0/1854061725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec105470 0x7f30ec105880 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 -- 172.21.15.50:0/1854061725 >> 172.21.15.50:0/1854061725 conn(0x7f30ec0ff9c0 msgr2=0x7f30ec101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 -- 172.21.15.50:0/1854061725 shutdown_connections 2024-05-10T21:07:45.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.167+0000 7f30f2200700 1 -- 172.21.15.50:0/1854061725 wait complete. 2024-05-10T21:07:45.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.168+0000 7f30f2200700 1 Processor -- start 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 -- start start 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105470 0x7f30ec19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec105e10 0x7f30ec1a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10fdb0 con 0x7f30ec104340 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10fc30 con 0x7f30ec105e10 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30f2200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10ff30 con 0x7f30ec105470 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30eb400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30eb400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45762/0 (socket says 172.21.15.50:45762) 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30eb400700 1 -- 172.21.15.50:0/421988880 learned_addr learned my addr 172.21.15.50:0/421988880 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.169+0000 7f30e2a00700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105470 0x7f30ec19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30ebe00700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec105e10 0x7f30ec1a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:45.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 -- 172.21.15.50:0/421988880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105470 msgr2=0x7f30ec19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:45.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105470 0x7f30ec19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 -- 172.21.15.50:0/421988880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec105e10 msgr2=0x7f30ec1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:45.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec105e10 0x7f30ec1a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 -- 172.21.15.50:0/421988880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1a4690 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.170+0000 7f30eb400700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00ea10 tx=0x7f30dc00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.171+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00cbd0 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.171+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1a4980 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.171+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30dc00cd70 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.171+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc018680 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.171+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1a4e90 con 0x7f30ec104340 2024-05-10T21:07:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.172+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec0512c0 con 0x7f30ec104340 2024-05-10T21:07:45.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.175+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f30dc01f020 con 0x7f30ec104340 2024-05-10T21:07:45.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.176+0000 7f30e8c00700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 0x7f30d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:45.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.177+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f30dc0e2720 con 0x7f30ec104340 2024-05-10T21:07:45.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.177+0000 7f30e2a00700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 0x7f30d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:45.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.178+0000 7f30e2a00700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 0x7f30d00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b780 tx=0x7f30d40060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:45.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.179+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30dc0aa410 con 0x7f30ec104340 2024-05-10T21:07:45.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.421+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f30ec1a50c0 con 0x7f30ec104340 2024-05-10T21:07:45.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.424+0000 7f30e8c00700 1 -- 172.21.15.50:0/421988880 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5228 (secure 0 0 0) 0x7f30dc0a9d10 con 0x7f30ec104340 2024-05-10T21:07:45.425 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 82 2024-05-10T21:07:45.425 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:07:45.425 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-05-10T20:54:44.451103+0000 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-05-10T21:04:56.878889+0000 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:crush_version 26 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:full_ratio 0.95 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:backfillfull_ratio 0.9 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:nearfull_ratio 0.85 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:require_min_compat_client luminous 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:min_compat_client luminous 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:require_osd_release squid 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:stretch_mode_enabled false 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T21:07:45.426 INFO:teuthology.orchestra.run.smithi050.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:max_osd 12 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6802/2398884233,v1:172.21.15.50:6803/2398884233] [v2:172.21.15.50:6804/2398884233,v1:172.21.15.50:6805/2398884233] exists,up 87ed2f28-e9c4-4c8e-a444-06e540010639 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/1268741901,v1:172.21.15.92:6801/1268741901] [v2:172.21.15.92:6802/1268741901,v1:172.21.15.92:6803/1268741901] exists,up afef3a38-83d4-4d62-9db8-065d439b4fda 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/1105819435,v1:172.21.15.142:6801/1105819435] [v2:172.21.15.142:6802/1105819435,v1:172.21.15.142:6803/1105819435] exists,up b00cad0c-573c-4648-9dee-c63912cde7a1 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 up in weight 1 up_from 25 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6810/1367886270,v1:172.21.15.50:6811/1367886270] [v2:172.21.15.50:6812/1367886270,v1:172.21.15.50:6813/1367886270] exists,up 0d3deb55-a3e3-4b6d-9a12-6a411e9004ff 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1024551638,v1:172.21.15.92:6809/1024551638] [v2:172.21.15.92:6810/1024551638,v1:172.21.15.92:6811/1024551638] exists,up 352735a0-8b3b-47b4-accb-d76af31a6279 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/3719494232,v1:172.21.15.142:6809/3719494232] [v2:172.21.15.142:6810/3719494232,v1:172.21.15.142:6811/3719494232] exists,up 23efb5de-d1d9-4654-ab57-81f4a6187db5 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 up in weight 1 up_from 42 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6818/2895143943,v1:172.21.15.50:6819/2895143943] [v2:172.21.15.50:6820/2895143943,v1:172.21.15.50:6821/2895143943] exists,up fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/477438803,v1:172.21.15.92:6817/477438803] [v2:172.21.15.92:6818/477438803,v1:172.21.15.92:6819/477438803] exists,up 93d87384-c4c0-4d95-b34e-bd0ff0680647 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 up in weight 1 up_from 54 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/2574916592,v1:172.21.15.142:6817/2574916592] [v2:172.21.15.142:6818/2574916592,v1:172.21.15.142:6819/2574916592] exists,up b945a33a-16a3-4281-a58f-7a066f04c7e6 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 up in weight 1 up_from 60 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6826/2893617399,v1:172.21.15.50:6827/2893617399] [v2:172.21.15.50:6828/2893617399,v1:172.21.15.50:6829/2893617399] exists,up 462243ae-aa14-4706-828a-a0d7247e5afc 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 up in weight 1 up_from 66 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/955638920,v1:172.21.15.92:6825/955638920] [v2:172.21.15.92:6826/955638920,v1:172.21.15.92:6827/955638920] exists,up 08409a23-50ab-4268-9ad6-c50879b535f7 2024-05-10T21:07:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/256997334,v1:172.21.15.142:6825/256997334] [v2:172.21.15.142:6826/256997334,v1:172.21.15.142:6827/256997334] exists,up c6ba8a0f-61ce-48d4-a456-18eca77cbdcf 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.10 [2,4] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.1f [8,1,7,11] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.1b [4,8,5,9] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.36 [4,9] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.38 [3,9] 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/3321516664 expires 2024-05-11T20:55:55.390388+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/3321516664 expires 2024-05-11T20:55:55.390388+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/484554811 expires 2024-05-11T20:55:19.361123+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/484554811 expires 2024-05-11T20:55:19.361123+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3150174588 expires 2024-05-11T20:55:19.361123+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/2515468284 expires 2024-05-11T20:55:19.361123+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/2292381409 expires 2024-05-11T20:55:55.390388+0000 2024-05-10T21:07:45.428 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/1334201717 expires 2024-05-11T20:55:55.390388+0000 2024-05-10T21:07:45.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/356575456 expires 2024-05-11T20:55:19.361123+0000 2024-05-10T21:07:45.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3564243559 expires 2024-05-11T20:55:55.390388+0000 2024-05-10T21:07:45.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.427+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 msgr2=0x7f30d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:45.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.427+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 0x7f30d00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b780 tx=0x7f30d40060c0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.427+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 msgr2=0x7f30ec19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.427+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00ea10 tx=0x7f30dc00ed20 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 shutdown_connections 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30d00b2880 0x7f30d00b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30ec105e10 0x7f30ec1a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30ec104340 0x7f30ec19cb30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 --2- 172.21.15.50:0/421988880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30ec105470 0x7f30ec19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 >> 172.21.15.50:0/421988880 conn(0x7f30ec0ff9c0 msgr2=0x7f30ec0759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 shutdown_connections 2024-05-10T21:07:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:45.428+0000 7f30f2200700 1 -- 172.21.15.50:0/421988880 wait complete. 2024-05-10T21:07:46.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:45 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/421988880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T21:07:46.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:45 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/421988880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T21:07:46.153 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:45 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/421988880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T21:07:46.228 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T21:07:46.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:07:46.735 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:47.073 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:46 smithi050 ceph-mon[29546]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:46 smithi092 ceph-mon[34415]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:47.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:46 smithi142 ceph-mon[34846]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:49.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:48 smithi050 ceph-mon[29546]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:49.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 -- 172.21.15.50:0/2263267694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72981034c0 msgr2=0x7f72981038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:49.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 --2- 172.21.15.50:0/2263267694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72981034c0 0x7f72981038d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f72880099c0 tx=0x7f7288009cd0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 -- 172.21.15.50:0/2263267694 shutdown_connections 2024-05-10T21:07:49.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 --2- 172.21.15.50:0/2263267694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7298108740 0x7f72981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 --2- 172.21.15.50:0/2263267694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72981034c0 0x7f72981038d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 --2- 172.21.15.50:0/2263267694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f7298108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.011+0000 7f729cc00700 1 -- 172.21.15.50:0/2263267694 >> 172.21.15.50:0/2263267694 conn(0x7f72980fedc0 msgr2=0x7f72981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.012+0000 7f729cc00700 1 -- 172.21.15.50:0/2263267694 shutdown_connections 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.012+0000 7f729cc00700 1 -- 172.21.15.50:0/2263267694 wait complete. 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.013+0000 7f729cc00700 1 Processor -- start 2024-05-10T21:07:49.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.013+0000 7f729cc00700 1 -- start start 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 0x7f72980ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f72980ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7298108740 0x7f72980ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7298113370 con 0x7f72981034c0 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f72981131f0 con 0x7f7298107df0 2024-05-10T21:07:49.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f729cc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72981134f0 con 0x7f7298108740 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f7295e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 0x7f72980ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f728d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f72980ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f7295e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 0x7f72980ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45780/0 (socket says 172.21.15.50:45780) 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.014+0000 7f7295e00700 1 -- 172.21.15.50:0/854375163 learned_addr learned my addr 172.21.15.50:0/854375163 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f7296800700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7298108740 0x7f72980ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 -- 172.21.15.50:0/854375163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7298108740 msgr2=0x7f72980ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:49.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7298108740 0x7f72980ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 -- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 msgr2=0x7f72980ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 0x7f72980ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 -- 172.21.15.50:0/854375163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72981a91d0 con 0x7f7298107df0 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.015+0000 7f728d400700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f72980ff7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f728000e9c0 tx=0x7f728000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.016+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728000c9c0 con 0x7f7298107df0 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.016+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728000cb60 con 0x7f7298107df0 2024-05-10T21:07:49.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.016+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72981a94c0 con 0x7f7298107df0 2024-05-10T21:07:49.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.016+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280018680 con 0x7f7298107df0 2024-05-10T21:07:49.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.016+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981a9a00 con 0x7f7298107df0 2024-05-10T21:07:49.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.017+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72980512c0 con 0x7f7298107df0 2024-05-10T21:07:49.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.019+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7280010430 con 0x7f7298107df0 2024-05-10T21:07:49.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.021+0000 7f728f400700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:49.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.021+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f72800e23b0 con 0x7f7298107df0 2024-05-10T21:07:49.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.021+0000 7f7295e00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:49.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.022+0000 7f7295e00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 0x7f72780b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7288003760 tx=0x7f728801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.026+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72800aa0a0 con 0x7f7298107df0 2024-05-10T21:07:49.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:48 smithi092 ceph-mon[34415]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:49.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:48 smithi142 ceph-mon[34846]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:49.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.317+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f729802e5f0 con 0x7f7298107df0 2024-05-10T21:07:49.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.318+0000 7f728f400700 1 -- 172.21.15.50:0/854375163 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7f72800b2120 con 0x7f7298107df0 2024-05-10T21:07:49.320 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:49.320 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 msgr2=0x7f72780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 0x7f72780b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7288003760 tx=0x7f728801a040 comp rx=0 tx=0).stop 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 msgr2=0x7f72980ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f72980ff7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f728000e9c0 tx=0x7f728000ecd0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 shutdown_connections 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7298108740 0x7f72980ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72981034c0 0x7f72980ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 --2- 172.21.15.50:0/854375163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7298107df0 0x7f72980ff7e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.322+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 >> 172.21.15.50:0/854375163 conn(0x7f72980fedc0 msgr2=0x7f72981063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:49.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.323+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 shutdown_connections 2024-05-10T21:07:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:49.323+0000 7f729cc00700 1 -- 172.21.15.50:0/854375163 wait complete. 2024-05-10T21:07:49.325 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:07:50.069 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:49 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/854375163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:50.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:49 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/854375163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:50.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:49 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/854375163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:50.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:07:50.637 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:51.114 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:50 smithi050 ceph-mon[29546]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:51.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:50 smithi092 ceph-mon[34415]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:51.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:50 smithi142 ceph-mon[34846]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:52.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 -- 172.21.15.50:0/1336574403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 msgr2=0x7f755c10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:52.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 --2- 172.21.15.50:0/1336574403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c10abd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75480098f0 tx=0x7f7548009c00 comp rx=0 tx=0).stop 2024-05-10T21:07:52.877 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:52 smithi050 ceph-mon[29546]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 -- 172.21.15.50:0/1336574403 shutdown_connections 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 --2- 172.21.15.50:0/1336574403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c10b110 0x7f755c117640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 --2- 172.21.15.50:0/1336574403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c1107a0 0x7f755c110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 --2- 172.21.15.50:0/1336574403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c10abd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 -- 172.21.15.50:0/1336574403 >> 172.21.15.50:0/1336574403 conn(0x7f755c076b00 msgr2=0x7f755c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 -- 172.21.15.50:0/1336574403 shutdown_connections 2024-05-10T21:07:52.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.876+0000 7f7561000700 1 -- 172.21.15.50:0/1336574403 wait complete. 2024-05-10T21:07:52.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.877+0000 7f7561000700 1 Processor -- start 2024-05-10T21:07:52.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 -- start start 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c1107a0 0x7f755c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f755c11a8f0 con 0x7f755c10b110 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f755c11a770 con 0x7f755c1107a0 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f7561000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f755c11aa70 con 0x7f755c10a7c0 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f755aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:52.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f755aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45800/0 (socket says 172.21.15.50:45800) 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f755aa00700 1 -- 172.21.15.50:0/1279938551 learned_addr learned my addr 172.21.15.50:0/1279938551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.878+0000 7f755b400700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755be00700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c1107a0 0x7f755c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 -- 172.21.15.50:0/1279938551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 msgr2=0x7f755c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 -- 172.21.15.50:0/1279938551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c1107a0 msgr2=0x7f755c1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c1107a0 0x7f755c1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 -- 172.21.15.50:0/1279938551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c1a61d0 con 0x7f755c10b110 2024-05-10T21:07:52.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.879+0000 7f755aa00700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7548009230 tx=0x7f754800f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.880+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548027070 con 0x7f755c10b110 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.880+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7548005570 con 0x7f755c10b110 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.880+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c1a6460 con 0x7f755c10b110 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.880+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548004930 con 0x7f755c10b110 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.880+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c1a6940 con 0x7f755c10b110 2024-05-10T21:07:52.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.881+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755c0512c0 con 0x7f755c10b110 2024-05-10T21:07:52.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.884+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f754802a070 con 0x7f755c10b110 2024-05-10T21:07:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.885+0000 7f7553e00700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 0x7f75400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.886+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f75480e2ff0 con 0x7f755c10b110 2024-05-10T21:07:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.886+0000 7f755b400700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 0x7f75400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:52.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.886+0000 7f755b400700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 0x7f75400b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f754c009750 tx=0x7f754c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:52.888+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7548031020 con 0x7f755c10b110 2024-05-10T21:07:53.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:52 smithi092 ceph-mon[34415]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:53.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:52 smithi142 ceph-mon[34846]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:07:53.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.178+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f755c02e5f0 con 0x7f755c10b110 2024-05-10T21:07:53.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.180+0000 7f7553e00700 1 -- 172.21.15.50:0/1279938551 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11852 (secure 0 0 0) 0x7f7548013030 con 0x7f755c10b110 2024-05-10T21:07:53.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:53.181 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:04:50.607977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 msgr2=0x7f75400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 0x7f75400b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f754c009750 tx=0x7f754c009310 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 msgr2=0x7f755c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7548009230 tx=0x7f754800f990 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 shutdown_connections 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75400b2880 0x7f75400b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f755c1107a0 0x7f755c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c10a7c0 0x7f755c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 --2- 172.21.15.50:0/1279938551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f755c10b110 0x7f755c1a18a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:53.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 >> 172.21.15.50:0/1279938551 conn(0x7f755c076b00 msgr2=0x7f755c108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:53.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 shutdown_connections 2024-05-10T21:07:53.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:53.183+0000 7f7561000700 1 -- 172.21.15.50:0/1279938551 wait complete. 2024-05-10T21:07:53.186 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 24 2024-05-10T21:07:53.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:07:54.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:53 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1279938551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:54.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:53 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1279938551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:54.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:53 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1279938551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:07:54.452 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:54.904 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:54 smithi050 ceph-mon[29546]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:55.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:54 smithi092 ceph-mon[34415]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:55.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:54 smithi142 ceph-mon[34846]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.754+0000 7f5c69200700 1 -- 172.21.15.50:0/3855351417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 msgr2=0x7f5c64113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.754+0000 7f5c69200700 1 --2- 172.21.15.50:0/3855351417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c64113200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009fc0 tx=0x7f5c5c00b330 comp rx=0 tx=0).stop 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.754+0000 7f5c69200700 1 -- 172.21.15.50:0/3855351417 shutdown_connections 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 --2- 172.21.15.50:0/3855351417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c64113200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 --2- 172.21.15.50:0/3855351417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c64106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 --2- 172.21.15.50:0/3855351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 -- 172.21.15.50:0/3855351417 >> 172.21.15.50:0/3855351417 conn(0x7f5c64102030 msgr2=0x7f5c64104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 -- 172.21.15.50:0/3855351417 shutdown_connections 2024-05-10T21:07:56.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.755+0000 7f5c69200700 1 -- 172.21.15.50:0/3855351417 wait complete. 2024-05-10T21:07:56.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.756+0000 7f5c69200700 1 Processor -- start 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.756+0000 7f5c69200700 1 -- start start 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c641a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641164d0 con 0x7f5c6410b8a0 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64116350 con 0x7f5c6410af50 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c69200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64116650 con 0x7f5c64106620 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c63e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c641a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c62a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c63400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:56.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c63400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56778/0 (socket says 172.21.15.50:56778) 2024-05-10T21:07:56.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c62a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33896/0 (socket says 172.21.15.50:33896) 2024-05-10T21:07:56.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.757+0000 7f5c63400700 1 -- 172.21.15.50:0/656495113 learned_addr learned my addr 172.21.15.50:0/656495113 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:07:56.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 -- 172.21.15.50:0/656495113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 msgr2=0x7f5c6419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:56.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 -- 172.21.15.50:0/656495113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 msgr2=0x7f5c6419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 -- 172.21.15.50:0/656495113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641a1cb0 con 0x7f5c6410b8a0 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63400700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c62a00700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.758+0000 7f5c63e00700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c641a1610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c015000 tx=0x7f5c5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.759+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c021070 con 0x7f5c6410b8a0 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.759+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c007ac0 con 0x7f5c6410b8a0 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.759+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c005040 con 0x7f5c6410b8a0 2024-05-10T21:07:56.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.759+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c641a1f40 con 0x7f5c6410b8a0 2024-05-10T21:07:56.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.759+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641a98c0 con 0x7f5c6410b8a0 2024-05-10T21:07:56.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.760+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c64107ad0 con 0x7f5c6410b8a0 2024-05-10T21:07:56.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.764+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c5c024070 con 0x7f5c6410b8a0 2024-05-10T21:07:56.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.766+0000 7f5c5be00700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 0x7f5c480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:07:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.766+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5c5c00f020 con 0x7f5c6410b8a0 2024-05-10T21:07:56.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.767+0000 7f5c63400700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 0x7f5c480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:07:56.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.767+0000 7f5c63400700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 0x7f5c480b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c54011550 tx=0x7f5c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:07:56.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:56.769+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c5c02b020 con 0x7f5c6410b8a0 2024-05-10T21:07:57.011 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:56 smithi050 ceph-mon[29546]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:07:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.010+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c6402e5f0 con 0x7f5c6410b8a0 2024-05-10T21:07:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.012+0000 7f5c5be00700 1 -- 172.21.15.50:0/656495113 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7f5c5c0ab730 con 0x7f5c6410b8a0 2024-05-10T21:07:57.013 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:07:57.013 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:07:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.015+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 msgr2=0x7f5c480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.015+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 0x7f5c480b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c54011550 tx=0x7f5c54009260 comp rx=0 tx=0).stop 2024-05-10T21:07:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.015+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 msgr2=0x7f5c641a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:07:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.015+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c641a1610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c015000 tx=0x7f5c5c00ccb0 comp rx=0 tx=0).stop 2024-05-10T21:07:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 shutdown_connections 2024-05-10T21:07:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5c480b2900 0x7f5c480b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c6410b8a0 0x7f5c641a1610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64106620 0x7f5c6419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 --2- 172.21.15.50:0/656495113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c6410af50 0x7f5c6419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:07:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 >> 172.21.15.50:0/656495113 conn(0x7f5c64102030 msgr2=0x7f5c6410a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:07:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 shutdown_connections 2024-05-10T21:07:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:07:57.016+0000 7f5c69200700 1 -- 172.21.15.50:0/656495113 wait complete. 2024-05-10T21:07:57.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:56 smithi092 ceph-mon[34415]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:07:57.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:56 smithi142 ceph-mon[34846]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:07:57.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T21:04:26.742871+0000', 'modified': '2024-05-10T21:04:50.607977+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24539, 'mds_1': 24520, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/1464775005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1464775005}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1464775005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/924361476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 924361476}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 924361476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6837/1752993045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1752993045}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1752993045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6839/3522449943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3522449943}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3522449943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.50:6841/908891204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6840', 'nonce': 908891204}, {'type': 'v1', 'addr': '172.21.15.50:6841', 'nonce': 908891204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6839/3200447854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3200447854}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3200447854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24539, 24502, 24520]} 2024-05-10T21:07:57.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T21:07:57.796 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T21:07:58.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:57 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/656495113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:58.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:57 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/656495113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:58.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:57 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/656495113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:07:58.279 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:07:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:07:58 smithi142 ceph-mon[34846]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:59.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:07:58 smithi050 ceph-mon[29546]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:07:59.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:07:58 smithi092 ceph-mon[34415]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:00.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3412065871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 msgr2=0x7f6d681049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3412065871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d681049f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d500099c0 tx=0x7f6d50009cd0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3412065871 shutdown_connections 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3412065871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68109860 0x7f6d681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3412065871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d681049f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3412065871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68108f10 0x7f6d68109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3412065871 >> 172.21.15.50:0/3412065871 conn(0x7f6d680ffe70 msgr2=0x7f6d681022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3412065871 shutdown_connections 2024-05-10T21:08:00.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.385+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3412065871 wait complete. 2024-05-10T21:08:00.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 Processor -- start 2024-05-10T21:08:00.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 -- start start 2024-05-10T21:08:00.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 0x7f6d6819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68109860 0x7f6d681a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68114500 con 0x7f6d681045e0 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68114380 con 0x7f6d68108f10 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.387+0000 7f6d6fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68114680 con 0x7f6d68109860 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 0x7f6d6819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58228/0 (socket says 172.21.15.50:58228) 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d6d200700 1 -- 172.21.15.50:0/3260727204 learned_addr learned my addr 172.21.15.50:0/3260727204 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68109860 0x7f6d681a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 -- 172.21.15.50:0/3260727204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68109860 msgr2=0x7f6d681a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68109860 0x7f6d681a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 -- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 msgr2=0x7f6d6819d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 -- 172.21.15.50:0/3260727204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d681a1f80 con 0x7f6d68108f10 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d6d200700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.388+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 0x7f6d6819d650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9c0 tx=0x7f6d5800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.389+0000 7f6d66000700 1 -- 172.21.15.50:0/3260727204 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004220 con 0x7f6d68108f10 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.389+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d681a2270 con 0x7f6d68108f10 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.389+0000 7f6d66000700 1 -- 172.21.15.50:0/3260727204 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580043c0 con 0x7f6d68108f10 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.389+0000 7f6d66000700 1 -- 172.21.15.50:0/3260727204 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005020 con 0x7f6d68108f10 2024-05-10T21:08:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.389+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d681a9b90 con 0x7f6d68108f10 2024-05-10T21:08:00.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.393+0000 7f6d66000700 1 -- 172.21.15.50:0/3260727204 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6d58004750 con 0x7f6d68108f10 2024-05-10T21:08:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 Processor -- start 2024-05-10T21:08:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 -- start start 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7ac0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559ef79e29d0 con 0x559ef7942df0 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ef7980710 con 0x559ef7ac67e0 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.394+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ef79de990 con 0x559ef7ac09a0 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d66000700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 0x7f6d480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33932/0 (socket says 172.21.15.50:33932) 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 -- 172.21.15.50:0/1454258222 learned_addr learned my addr 172.21.15.50:0/1454258222 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7ac0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d66000700 1 -- 172.21.15.50:0/3260727204 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6d580e2100 con 0x7f6d68108f10 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 -- 172.21.15.50:0/1454258222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 msgr2=0x559ef7ac0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7ac0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 -- 172.21.15.50:0/1454258222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 msgr2=0x559ef7ac62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6d200700 1 -- 172.21.15.50:0/1454258222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ef792f880 con 0x559ef7ac67e0 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.395+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 0x7f6d480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.396+0000 7f6d6d200700 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005910 tx=0x7f6d50005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0257536160595d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.396+0000 7f6d5e000700 1 -- 172.21.15.50:0/1454258222 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50027070 con 0x559ef7ac67e0 2024-05-10T21:08:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.396+0000 7f6d5e000700 1 -- 172.21.15.50:0/1454258222 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d50005560 con 0x559ef7ac67e0 2024-05-10T21:08:00.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.396+0000 7f6d5e000700 1 -- 172.21.15.50:0/1454258222 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003e00 con 0x559ef7ac67e0 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1454258222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 msgr2=0x559ef7ac0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005910 tx=0x7f6d50005040 comp rx=0 tx=0).stop 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 0x7f6d480b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009ec0 tx=0x7f6d60009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1454258222 shutdown_connections 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7ac0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1454258222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac67e0 0x559ef7ac0460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1454258222 >> 172.21.15.50:0/1454258222 conn(0x559ef7987f20 msgr2=0x559ef7abb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1454258222 shutdown_connections 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.397+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1454258222 wait complete. 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.398+0000 7f6d7e7fab80 1 -- start start 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.398+0000 7f6d7e7fab80 1 Processor -- start 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 0x559ef7aca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac67e0 0x559ef7aca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559ef7acd260 con 0x559ef7942df0 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ef7948ad0 con 0x559ef7ac09a0 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ef7acd3e0 con 0x559ef7ac67e0 2024-05-10T21:08:00.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d7e7fab80 10 client.0 _ll_register_callbacks cb 0x559ef7989c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58254/0 (socket says 172.21.15.50:58254) 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.399+0000 7f6d6d200700 1 -- 172.21.15.50:0/4032706969 learned_addr learned my addr 172.21.15.50:0/4032706969 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 0x559ef7aca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac67e0 0x559ef7aca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 -- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac67e0 msgr2=0x559ef7aca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac67e0 0x559ef7aca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 -- 172.21.15.50:0/4032706969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 msgr2=0x559ef7ac9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 -- 172.21.15.50:0/4032706969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ef79a83d0 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 0x559ef7aca470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800c5b0 tx=0x7f6d5801dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdab28f209a2cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.400+0000 7f6d56a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800c760 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d50005560 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6d5801dd60 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d580a9200 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d7e7fab80 10 client.25001 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ef7ad7dc0 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d50005560 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6d580b20c0 con 0x559ef7ac09a0 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.401+0000 7f6d7e7fab80 10 client.25001 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:00.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d7e7fab80 10 client.25001 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:08:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d7e7fab80 20 client.25001 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7ada370 0x559ef7adb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d7e7fab80 4 client.25001 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ef7a8d170 con 0x559ef7ada370 2024-05-10T21:08:00.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.402+0000 7f6d6d200700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7ada370 0x559ef7adb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.403+0000 7f6d6d200700 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7ada370 0x559ef7adb7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=febbfc5154787b7d server_cookie=8ba46ff402d49350 in_seq=0 out_seq=0 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.403+0000 7f6d56a00700 10 client.25001 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d56a00700 1 -- 172.21.15.50:0/4032706969 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ef7a8d170 con 0x559ef7ada370 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d56a00700 10 client.25001 handle_command_reply: tid=0 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 client.25001 shutdown 2024-05-10T21:08:00.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 msgr2=0x559ef7aca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 0x559ef7aca470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800c5b0 tx=0x7f6d5801dcf0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 shutdown_connections 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7ada370 0x559ef7adb7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac67e0 0x559ef7aca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7942df0 0x559ef7ac9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/4032706969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7ac09a0 0x559ef7aca470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d56a00700 0 client.25001 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.404+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 >> 172.21.15.50:0/4032706969 conn(0x559ef7987f20 msgr2=0x559ef7abf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.405+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 shutdown_connections 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.405+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/4032706969 wait complete. 2024-05-10T21:08:00.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.405+0000 7f6d7e7fab80 20 client.25001 trim_cache size 0 max 16384 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 Processor -- start 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 -- start start 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7ae4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559ef7ad9b20 con 0x559ef7ac67e0 2024-05-10T21:08:00.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ef7ada1f0 con 0x559ef7942df0 2024-05-10T21:08:00.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.428+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ef7ada8c0 con 0x559ef7ac09a0 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56832/0 (socket says 172.21.15.50:56832) 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 -- 172.21.15.50:0/1207180043 learned_addr learned my addr 172.21.15.50:0/1207180043 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d6d200700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7ae4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 -- 172.21.15.50:0/1207180043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 msgr2=0x559ef7ac0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 -- 172.21.15.50:0/1207180043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 msgr2=0x559ef7ae4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7ae4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d5fe00700 1 -- 172.21.15.50:0/1207180043 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ef7ada010 con 0x559ef7ac09a0 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.429+0000 7f6d6dc00700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7ae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.430+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d580a5770 tx=0x7f6d5800c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13405b14ff366878 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.430+0000 7f6d5d600700 1 -- 172.21.15.50:0/1207180043 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d580a9df0 con 0x559ef7ac09a0 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.430+0000 7f6d5d600700 1 -- 172.21.15.50:0/1207180043 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580adaa0 con 0x559ef7ac09a0 2024-05-10T21:08:00.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.430+0000 7f6d5d600700 1 -- 172.21.15.50:0/1207180043 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58091310 con 0x559ef7ac09a0 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1207180043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 msgr2=0x559ef7aca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d580a5770 tx=0x7f6d5800c930 comp rx=0 tx=0).stop 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1207180043 shutdown_connections 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7ae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/1207180043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca7f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1207180043 >> 172.21.15.50:0/1207180043 conn(0x559ef7987f20 msgr2=0x559ef7ac8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1207180043 shutdown_connections 2024-05-10T21:08:00.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.431+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/1207180043 wait complete. 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.432+0000 7f6d7e7fab80 1 -- start start 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.432+0000 7f6d7e7fab80 1 Processor -- start 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7aca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559ef7adb660 con 0x559ef7ac67e0 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ef7adaf90 con 0x559ef7942df0 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ef7adbd30 con 0x559ef7ac09a0 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d7e7fab80 10 client.0 _ll_register_callbacks cb 0x559ef7989c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7aca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7aca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58272/0 (socket says 172.21.15.50:58272) 2024-05-10T21:08:00.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56848/0 (socket says 172.21.15.50:56848) 2024-05-10T21:08:00.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6dc00700 1 -- 172.21.15.50:0/9339907 learned_addr learned my addr 172.21.15.50:0/9339907 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:00.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33958/0 (socket says 172.21.15.50:33958) 2024-05-10T21:08:00.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 -- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 msgr2=0x559ef7aca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 -- 172.21.15.50:0/9339907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 msgr2=0x559ef7aca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7aca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.433+0000 7f6d6d200700 1 -- 172.21.15.50:0/9339907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ef7ae2f60 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d6d200700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005b40 tx=0x7f6d50005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559c6a3c49950967 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d56a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50027040 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d580adaa0 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.434+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6d50003c50 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d50003e50 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 10 client.25007 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ef7b45960 con 0x559ef7942df0 2024-05-10T21:08:00.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580adaa0 con 0x559ef7942df0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6d50021030 con 0x559ef7942df0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 10 client.25007 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 10 client.25007 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 20 client.25007 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7b478f0 0x559ef7b48d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 4 client.25007 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.435+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "status"}) -- 0x559ef79f03b0 con 0x559ef7b478f0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.436+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7b478f0 0x559ef7b48d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.436+0000 7f6d56a00700 10 client.25007 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.436+0000 7f6d5fe00700 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7b478f0 0x559ef7b48d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=760c3b3c61dad652 server_cookie=9d0d686ac2515c92 in_seq=0 out_seq=0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d56a00700 1 -- 172.21.15.50:0/9339907 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559ef79f03b0 con 0x559ef7b478f0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d56a00700 10 client.25007 handle_command_reply: tid=0 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 client.25007 shutdown 2024-05-10T21:08:00.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 msgr2=0x559ef7ac9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005b40 tx=0x7f6d50005fb0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 shutdown_connections 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x559ef7b478f0 0x559ef7b48d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ef7942df0 0x559ef7ac9c00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559ef7ac67e0 0x559ef7aca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 --2- 172.21.15.50:0/9339907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ef7ac09a0 0x559ef7aca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 >> 172.21.15.50:0/9339907 conn(0x559ef7987f20 msgr2=0x559ef7ac5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.437+0000 7f6d56a00700 0 client.25007 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 shutdown_connections 2024-05-10T21:08:00.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d7e7fab80 1 -- 172.21.15.50:0/9339907 wait complete. 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d7e7fab80 20 client.25007 trim_cache size 0 max 16384 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24502, 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:08:00.442 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 196.636218937, 2024-05-10T21:08:00.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:08:00.443 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:08:00.443 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:08:00.443 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 247.557778692 2024-05-10T21:08:00.443 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 msgr2=0x7f6d480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 0x7f6d480b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009ec0 tx=0x7f6d60009040 comp rx=0 tx=0).stop 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 msgr2=0x7f6d6819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.438+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 0x7f6d6819d650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9c0 tx=0x7f6d5800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 shutdown_connections 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6d480b2900 0x7f6d480b4e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d68109860 0x7f6d681a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d681045e0 0x7f6d6819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 --2- 172.21.15.50:0/3260727204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68108f10 0x7f6d6819d650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:00.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 >> 172.21.15.50:0/3260727204 conn(0x7f6d680ffe70 msgr2=0x7f6d68107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:00.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 shutdown_connections 2024-05-10T21:08:00.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:00.439+0000 7f6d6fc00700 1 -- 172.21.15.50:0/3260727204 wait complete. 2024-05-10T21:08:01.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:00 smithi142 ceph-mon[34846]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:01.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:00 smithi050 ceph-mon[29546]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:01.357 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.a status 2024-05-10T21:08:01.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:00 smithi092 ceph-mon[34415]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:01.829 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:03.045 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:02 smithi050 ceph-mon[29546]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:03.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:02 smithi142 ceph-mon[34846]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:03.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:02 smithi092 ceph-mon[34415]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 -- 172.21.15.50:0/1821058263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab8108f20 msgr2=0x7f0ab8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 --2- 172.21.15.50:0/1821058263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab8108f20 0x7f0ab8109330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00099c0 tx=0x7f0aa0009cd0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 -- 172.21.15.50:0/1821058263 shutdown_connections 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 --2- 172.21.15.50:0/1821058263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8109870 0x7f0ab81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 --2- 172.21.15.50:0/1821058263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab81045f0 0x7f0ab8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 --2- 172.21.15.50:0/1821058263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab8108f20 0x7f0ab8109330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 -- 172.21.15.50:0/1821058263 >> 172.21.15.50:0/1821058263 conn(0x7f0ab8100040 msgr2=0x7f0ab8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 -- 172.21.15.50:0/1821058263 shutdown_connections 2024-05-10T21:08:03.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.990+0000 7f0abce00700 1 -- 172.21.15.50:0/1821058263 wait complete. 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.991+0000 7f0abce00700 1 Processor -- start 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.991+0000 7f0abce00700 1 -- start start 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 0x7f0ab819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab8109870 0x7f0ab81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab81144d0 con 0x7f0ab81045f0 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8114350 con 0x7f0ab8108f20 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0abce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8114650 con 0x7f0ab8109870 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53094/0 (socket says 172.21.15.50:53094) 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0ab5e00700 1 -- 172.21.15.50:0/2755708122 learned_addr learned my addr 172.21.15.50:0/2755708122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0aad400700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 0x7f0ab819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:03.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.992+0000 7f0ab6800700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab8109870 0x7f0ab81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:03.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 -- 172.21.15.50:0/2755708122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab8109870 msgr2=0x7f0ab81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:03.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab8109870 0x7f0ab81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 -- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 msgr2=0x7f0ab819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:03.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 -- 172.21.15.50:0/2755708122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab81a1f50 con 0x7f0ab8108f20 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.993+0000 7f0aad400700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 0x7f0ab819d620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098e0 tx=0x7f0aa0005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.994+0000 7f0aaf400700 1 -- 172.21.15.50:0/2755708122 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0026070 con 0x7f0ab8108f20 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.994+0000 7f0aaf400700 1 -- 172.21.15.50:0/2755708122 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa00056c0 con 0x7f0ab8108f20 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.994+0000 7f0aaf400700 1 -- 172.21.15.50:0/2755708122 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000f960 con 0x7f0ab8108f20 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.994+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab81a21e0 con 0x7f0ab8108f20 2024-05-10T21:08:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.994+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab81a9b60 con 0x7f0ab8108f20 2024-05-10T21:08:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.997+0000 7f0aaf400700 1 -- 172.21.15.50:0/2755708122 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0aa0029070 con 0x7f0ab8108f20 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.998+0000 7f0acb8ceb80 1 Processor -- start 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 -- start start 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0aaf400700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 0x7f0a9c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e204d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e204e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595e1f6f9d0 con 0x5595e204dc00 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595e1f0d710 con 0x5595e2053ba0 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595e1f6b990 con 0x5595e1ecfdf0 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:03.999+0000 7f0aaf400700 1 -- 172.21.15.50:0/2755708122 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0aa00e2ab0 con 0x7f0ab8108f20 2024-05-10T21:08:04.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34704/0 (socket says 172.21.15.50:34704) 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 -- 172.21.15.50:0/421616304 learned_addr learned my addr 172.21.15.50:0/421616304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0aad400700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e204d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab6800700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e204e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 -- 172.21.15.50:0/421616304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 msgr2=0x5595e204d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e204d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 -- 172.21.15.50:0/421616304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 msgr2=0x5595e204e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e204e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 -- 172.21.15.50:0/421616304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e1ebc880 con 0x5595e1ecfdf0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab6800700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e204e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab6800700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 0x7f0a9c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.000+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80117e0 tx=0x7f0aa8011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6112cbc4adb6201e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.001+0000 7f0aa7400700 1 -- 172.21.15.50:0/421616304 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801e070 con 0x5595e1ecfdf0 2024-05-10T21:08:04.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.001+0000 7f0aa7400700 1 -- 172.21.15.50:0/421616304 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa800f930 con 0x5595e1ecfdf0 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.001+0000 7f0aa7400700 1 -- 172.21.15.50:0/421616304 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80198e0 con 0x5595e1ecfdf0 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.001+0000 7f0ab6800700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 0x7f0a9c0b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009550 tx=0x7f0ab000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/421616304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 msgr2=0x5595e2053660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80117e0 tx=0x7f0aa8011af0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/421616304 shutdown_connections 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e204e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2053660 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/421616304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e204d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/421616304 >> 172.21.15.50:0/421616304 conn(0x5595e1f0ea30 msgr2=0x5595e2048740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/421616304 shutdown_connections 2024-05-10T21:08:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.002+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/421616304 wait complete. 2024-05-10T21:08:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.003+0000 7f0acb8ceb80 1 -- start start 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.003+0000 7f0acb8ceb80 1 Processor -- start 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2057330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e2057db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595e205a670 con 0x5595e204dc00 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595e1ed5ad0 con 0x5595e2053ba0 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595e205a7f0 con 0x5595e1ecfdf0 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0acb8ceb80 10 client.0 _ll_register_callbacks cb 0x5595e1f03330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0aad400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0aad400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53110/0 (socket says 172.21.15.50:53110) 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0aad400700 1 -- 172.21.15.50:0/3635619258 learned_addr learned my addr 172.21.15.50:0/3635619258 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0ab6800700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e2057db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.004+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2057330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 -- 172.21.15.50:0/3635619258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 msgr2=0x5595e2057330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2057330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 -- 172.21.15.50:0/3635619258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 msgr2=0x5595e2057db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e2057db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 -- 172.21.15.50:0/3635619258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e1f353d0 con 0x5595e204dc00 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0ab6800700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e2057db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2057330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0aad400700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5def30258cc8c3f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0a9be00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00ae4e0 con 0x5595e204dc00 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0aa00aa390 con 0x5595e204dc00 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.005+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aa800f930 con 0x5595e204dc00 2024-05-10T21:08:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0aa00b2f00 con 0x5595e204dc00 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 10 client.14892 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5595e20651c0 con 0x5595e204dc00 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa800f930 con 0x5595e204dc00 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0aa00ae130 con 0x5595e204dc00 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 10 client.14892 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 10 client.14892 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 20 client.14892 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20678a0 0x5595e2068d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 4 client.14892 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.006+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e201a020 con 0x5595e20678a0 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.007+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20678a0 0x5595e2068d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.007+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20678a0 0x5595e2068d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a50e43a182bde7ae server_cookie=9ca11959b68cce39 in_seq=0 out_seq=0 2024-05-10T21:08:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.007+0000 7f0a9be00700 10 client.14892 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:08:04.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.008+0000 7f0a9be00700 1 -- 172.21.15.50:0/3635619258 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595e201a020 con 0x5595e20678a0 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.008+0000 7f0a9be00700 10 client.14892 handle_command_reply: tid=0 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.008+0000 7f0acb8ceb80 1 client.14892 shutdown 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 msgr2=0x5595e2057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0000c00 tx=0x7f0aa0009ef0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 shutdown_connections 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e2053ba0 0x5595e2057db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20678a0 0x5595e2068d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e1ecfdf0 0x5595e2057330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/3635619258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e204dc00 0x5595e2057870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 >> 172.21.15.50:0/3635619258 conn(0x5595e1f0ea30 msgr2=0x5595e204c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0a9be00700 0 client.14892 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 shutdown_connections 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/3635619258 wait complete. 2024-05-10T21:08:04.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.009+0000 7f0acb8ceb80 20 client.14892 trim_cache size 0 max 16384 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 Processor -- start 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 -- start start 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e204d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e2063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595e2067050 con 0x5595e1ecfdf0 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595e2067720 con 0x5595e204dc00 2024-05-10T21:08:04.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.033+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595e2067df0 con 0x5595e2053ba0 2024-05-10T21:08:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e204d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34716/0 (socket says 172.21.15.50:34716) 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 -- 172.21.15.50:0/1651068739 learned_addr learned my addr 172.21.15.50:0/1651068739 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0aad400700 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e2063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 -- 172.21.15.50:0/1651068739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 msgr2=0x5595e2063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e2063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 -- 172.21.15.50:0/1651068739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 msgr2=0x5595e204d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e204d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 -- 172.21.15.50:0/1651068739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e2042ab0 con 0x5595e2053ba0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.034+0000 7f0ab6800700 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0017ae0 tx=0x7f0ab0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b874ed12271fc93f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.035+0000 7f0aa6a00700 1 -- 172.21.15.50:0/1651068739 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0028070 con 0x5595e2053ba0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.035+0000 7f0aa6a00700 1 -- 172.21.15.50:0/1651068739 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab0015770 con 0x5595e2053ba0 2024-05-10T21:08:04.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.035+0000 7f0aa6a00700 1 -- 172.21.15.50:0/1651068739 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00043b0 con 0x5595e2053ba0 2024-05-10T21:08:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/1651068739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 msgr2=0x5595e20562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0017ae0 tx=0x7f0ab0009920 comp rx=0 tx=0).stop 2024-05-10T21:08:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/1651068739 shutdown_connections 2024-05-10T21:08:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e20562b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e204d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/1651068739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e2063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/1651068739 >> 172.21.15.50:0/1651068739 conn(0x5595e1f0ea30 msgr2=0x5595e2047e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/1651068739 shutdown_connections 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/1651068739 wait complete. 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.036+0000 7f0acb8ceb80 1 -- start start 2024-05-10T21:08:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 Processor -- start 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e2062e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e20633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595e2068b90 con 0x5595e1ecfdf0 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595e20684c0 con 0x5595e204dc00 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595e2069260 con 0x5595e2053ba0 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.037+0000 7f0acb8ceb80 10 client.0 _ll_register_callbacks cb 0x5595e1f03330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e2062e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34724/0 (socket says 172.21.15.50:34724) 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 -- 172.21.15.50:0/400685403 learned_addr learned my addr 172.21.15.50:0/400685403 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0aad400700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e20633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 -- 172.21.15.50:0/400685403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 msgr2=0x5595e20633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e20633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 -- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 msgr2=0x5595e2062e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e2062e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.038+0000 7f0ab6800700 1 -- 172.21.15.50:0/400685403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e2063c30 con 0x5595e2053ba0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0aad400700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e20633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e2062e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0ab6800700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009110 tx=0x7f0ab0015fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b225af26cac8c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0a9be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0028040 con 0x5595e2053ba0 2024-05-10T21:08:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab0015770 con 0x5595e2053ba0 2024-05-10T21:08:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0ab0003b80 con 0x5595e2053ba0 2024-05-10T21:08:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.039+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0ab0003d80 con 0x5595e2053ba0 2024-05-10T21:08:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0acb8ceb80 10 client.25015 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5595e20d3240 con 0x5595e2053ba0 2024-05-10T21:08:04.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab0015770 con 0x5595e2053ba0 2024-05-10T21:08:04.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0ab0038460 con 0x5595e2053ba0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0acb8ceb80 10 client.25015 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0acb8ceb80 10 client.25015 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.040+0000 7f0acb8ceb80 20 client.25015 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20d54a0 0x5595e20d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0acb8ceb80 4 client.25015 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "status"}) -- 0x5595e1f7d3b0 con 0x5595e20d54a0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20d54a0 0x5595e20d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0ab5e00700 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20d54a0 0x5595e20d6910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=505ec24838d059d9 server_cookie=69c96719a3fb8ea0 in_seq=0 out_seq=0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.041+0000 7f0a9be00700 10 client.25015 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0a9be00700 1 -- 172.21.15.50:0/400685403 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5595e1f7d3b0 con 0x5595e20d54a0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0a9be00700 10 client.25015 handle_command_reply: tid=0 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 client.25015 shutdown 2024-05-10T21:08:04.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 msgr2=0x5595e2070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009110 tx=0x7f0ab0015fd0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 shutdown_connections 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5595e20d54a0 0x5595e20d6910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595e2053ba0 0x5595e2070790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595e1ecfdf0 0x5595e2062e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 --2- 172.21.15.50:0/400685403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595e204dc00 0x5595e20633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0a9be00700 0 client.25015 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 >> 172.21.15.50:0/400685403 conn(0x5595e1f0ea30 msgr2=0x5595e2052c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 shutdown_connections 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 1 -- 172.21.15.50:0/400685403 wait complete. 2024-05-10T21:08:04.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.042+0000 7f0acb8ceb80 20 client.25015 trim_cache size 0 max 16384 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24520, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 201.250375037, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 242.200714621 2024-05-10T21:08:04.046 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:08:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 msgr2=0x7f0a9c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 0x7f0a9c0b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009550 tx=0x7f0ab000b040 comp rx=0 tx=0).stop 2024-05-10T21:08:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 msgr2=0x7f0ab819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 0x7f0ab819d620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00098e0 tx=0x7f0aa0005280 comp rx=0 tx=0).stop 2024-05-10T21:08:04.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 shutdown_connections 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a9c0b2900 0x7f0a9c0b4e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ab8109870 0x7f0ab81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ab81045f0 0x7f0ab819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 --2- 172.21.15.50:0/2755708122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ab8108f20 0x7f0ab819d620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.043+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 >> 172.21.15.50:0/2755708122 conn(0x7f0ab8100040 msgr2=0x7f0ab810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.044+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 shutdown_connections 2024-05-10T21:08:04.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:04.044+0000 7f0abce00700 1 -- 172.21.15.50:0/2755708122 wait complete. 2024-05-10T21:08:05.003 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T21:08:05.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:04 smithi050 ceph-mon[29546]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:05.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:04 smithi092 ceph-mon[34415]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:05.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:04 smithi142 ceph-mon[34846]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:05.465 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:07.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:06 smithi050 ceph-mon[29546]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:07.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:06 smithi092 ceph-mon[34415]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:07.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:06 smithi142 ceph-mon[34846]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:07.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 -- 172.21.15.50:0/443758036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba80764b0 msgr2=0x7f1ba8114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 --2- 172.21.15.50:0/443758036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba80764b0 0x7f1ba8114e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009990 tx=0x7f1b90009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 -- 172.21.15.50:0/443758036 shutdown_connections 2024-05-10T21:08:07.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 --2- 172.21.15.50:0/443758036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 0x7f1ba8117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 --2- 172.21.15.50:0/443758036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba8075b60 0x7f1ba8075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 --2- 172.21.15.50:0/443758036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba80764b0 0x7f1ba8114e30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 -- 172.21.15.50:0/443758036 >> 172.21.15.50:0/443758036 conn(0x7f1ba8100040 msgr2=0x7f1ba8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 -- 172.21.15.50:0/443758036 shutdown_connections 2024-05-10T21:08:07.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.783+0000 7f1baec00700 1 -- 172.21.15.50:0/443758036 wait complete. 2024-05-10T21:08:07.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.784+0000 7f1baec00700 1 Processor -- start 2024-05-10T21:08:07.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 -- start start 2024-05-10T21:08:07.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba8075b60 0x7f1ba81a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 0x7f1ba81a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811aac0 con 0x7f1ba8075b60 2024-05-10T21:08:07.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811a940 con 0x7f1ba8115370 2024-05-10T21:08:07.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1baec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba811ac40 con 0x7f1ba80764b0 2024-05-10T21:08:07.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.785+0000 7f1b9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34742/0 (socket says 172.21.15.50:34742) 2024-05-10T21:08:07.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 -- 172.21.15.50:0/2603527433 learned_addr learned my addr 172.21.15.50:0/2603527433 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:07.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1bacc00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 0x7f1ba81a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba8075b60 0x7f1ba81a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 -- 172.21.15.50:0/2603527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 msgr2=0x7f1ba81a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 0x7f1ba81a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 -- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba8075b60 msgr2=0x7f1ba81a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba8075b60 0x7f1ba81a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 -- 172.21.15.50:0/2603527433 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba81a6320 con 0x7f1ba80764b0 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.786+0000 7f1b9f400700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000c00 tx=0x7f1b900057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.787+0000 7f1ba5600700 1 -- 172.21.15.50:0/2603527433 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90027070 con 0x7f1ba80764b0 2024-05-10T21:08:07.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.787+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba81a65b0 con 0x7f1ba80764b0 2024-05-10T21:08:07.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.787+0000 7f1ba5600700 1 -- 172.21.15.50:0/2603527433 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90022470 con 0x7f1ba80764b0 2024-05-10T21:08:07.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.787+0000 7f1ba5600700 1 -- 172.21.15.50:0/2603527433 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90017650 con 0x7f1ba80764b0 2024-05-10T21:08:07.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.787+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81adf30 con 0x7f1ba80764b0 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.791+0000 7f1ba5600700 1 -- 172.21.15.50:0/2603527433 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1b9002a070 con 0x7f1ba80764b0 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 Processor -- start 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 -- start start 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aa6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5617519c79d0 con 0x561751927df0 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561751965710 con 0x561751aa5c00 2024-05-10T21:08:07.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.792+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5617519c3990 con 0x561751aabba0 2024-05-10T21:08:07.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1bacc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1bacc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53158/0 (socket says 172.21.15.50:53158) 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1bacc00700 1 -- 172.21.15.50:0/449596172 learned_addr learned my addr 172.21.15.50:0/449596172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1ba5600700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 0x7f1b8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1b9f400700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aa6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.793+0000 7f1ba7e00700 1 -- 172.21.15.50:0/449596172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 msgr2=0x561751aa6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aa6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba7e00700 1 -- 172.21.15.50:0/449596172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 msgr2=0x561751aab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba7e00700 1 -- 172.21.15.50:0/449596172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561751914880 con 0x561751aabba0 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba5600700 1 -- 172.21.15.50:0/2603527433 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1b900e2fd0 con 0x7f1ba80764b0 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1b9f400700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aa6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:08:07.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1bacc00700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:07.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1bacc00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 0x7f1b8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa56c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009310 tx=0x7f1b98009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47c622f023283998 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1bacc00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 0x7f1b8c0b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x561751aa69e0 tx=0x7f1ba0004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.794+0000 7f1b9d600700 1 -- 172.21.15.50:0/449596172 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98019020 con 0x561751aabba0 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.795+0000 7f1b9d600700 1 -- 172.21.15.50:0/449596172 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b98015af0 con 0x561751aabba0 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.795+0000 7f1b9d600700 1 -- 172.21.15.50:0/449596172 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98014a80 con 0x561751aabba0 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.795+0000 7f1bbd66db80 1 -- 172.21.15.50:0/449596172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 msgr2=0x561751aa56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.795+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa56c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009310 tx=0x7f1b98009920 comp rx=0 tx=0).stop 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 -- 172.21.15.50:0/449596172 shutdown_connections 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aa6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/449596172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa56c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 -- 172.21.15.50:0/449596172 >> 172.21.15.50:0/449596172 conn(0x561751966a30 msgr2=0x561751aa0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 -- 172.21.15.50:0/449596172 shutdown_connections 2024-05-10T21:08:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 -- 172.21.15.50:0/449596172 wait complete. 2024-05-10T21:08:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.796+0000 7f1bbd66db80 1 -- start start 2024-05-10T21:08:07.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 Processor -- start 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aaf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aabba0 0x561751aafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561751ab2670 con 0x561751aabba0 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56175192dad0 con 0x561751aa5c00 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561751ab27f0 con 0x561751927df0 2024-05-10T21:08:07.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1bbd66db80 10 client.0 _ll_register_callbacks cb 0x56175195b330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.797+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34754/0 (socket says 172.21.15.50:34754) 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1ba7e00700 1 -- 172.21.15.50:0/1614182044 learned_addr learned my addr 172.21.15.50:0/1614182044 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aaf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1bacc00700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aabba0 0x561751aafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 -- 172.21.15.50:0/1614182044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 msgr2=0x561751aaf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 -- 172.21.15.50:0/1614182044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aabba0 msgr2=0x561751aafd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aabba0 0x561751aafd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 -- 172.21.15.50:0/1614182044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175198d3d0 con 0x561751aa5c00 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b9f400700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aaf830 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1b900a78c0 tx=0x7f1b90009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd4d343f3a13bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.798+0000 7f1b96000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:07.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a9fa0 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1b900aaab0 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b98015af0 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b900a9dc0 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1bbd66db80 10 client.25034 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561751abd220 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b98015af0 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.799+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1b900b2f90 con 0x561751aa5c00 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 10 client.25034 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 10 client.25034 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 20 client.25034 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751abf850 0x561751ac0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 4 client.25034 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:07.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561751a72020 con 0x561751abf850 2024-05-10T21:08:07.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.800+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751abf850 0x561751ac0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.801+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751abf850 0x561751ac0cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d04bba86f673047 server_cookie=27c29d2294aabf8e in_seq=0 out_seq=0 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.801+0000 7f1b96000700 10 client.25034 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1b96000700 1 -- 172.21.15.50:0/1614182044 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561751a72020 con 0x561751abf850 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1b96000700 10 client.25034 handle_command_reply: tid=0 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 client.25034 shutdown 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 msgr2=0x561751aaf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aaf830 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1b900a78c0 tx=0x7f1b90009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 shutdown_connections 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751abf850 0x561751ac0cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aabba0 0x561751aafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751927df0 0x561751aaf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1614182044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aa5c00 0x561751aaf830 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 >> 172.21.15.50:0/1614182044 conn(0x561751966a30 msgr2=0x561751aa46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1b96000700 0 client.25034 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:07.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 shutdown_connections 2024-05-10T21:08:07.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.802+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1614182044 wait complete. 2024-05-10T21:08:07.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.803+0000 7f1bbd66db80 20 client.25034 trim_cache size 0 max 16384 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 Processor -- start 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 -- start start 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aa5c00 0x561751aaeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 0x561751abbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561751abf000 con 0x561751927df0 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561751abf6d0 con 0x561751aabba0 2024-05-10T21:08:07.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.826+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561751abfda0 con 0x561751aa5c00 2024-05-10T21:08:07.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53174/0 (socket says 172.21.15.50:53174) 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 -- 172.21.15.50:0/1845939988 learned_addr learned my addr 172.21.15.50:0/1845939988 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1b9f400700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aa5c00 0x561751aaeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1bacc00700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 0x561751abbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 -- 172.21.15.50:0/1845939988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aa5c00 msgr2=0x561751aaeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aa5c00 0x561751aaeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 -- 172.21.15.50:0/1845939988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 msgr2=0x561751abbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 0x561751abbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1ba7e00700 1 -- 172.21.15.50:0/1845939988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561751ab27f0 con 0x561751927df0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.827+0000 7f1bacc00700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 0x561751abbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.828+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b980145b0 tx=0x7f1b98009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65631b318591e10d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.828+0000 7f1b9cc00700 1 -- 172.21.15.50:0/1845939988 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98019040 con 0x561751927df0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.828+0000 7f1b9cc00700 1 -- 172.21.15.50:0/1845939988 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9802ce50 con 0x561751927df0 2024-05-10T21:08:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.828+0000 7f1b9cc00700 1 -- 172.21.15.50:0/1845939988 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98014b60 con 0x561751927df0 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1845939988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 msgr2=0x561751aa56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b980145b0 tx=0x7f1b98009c90 comp rx=0 tx=0).stop 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1845939988 shutdown_connections 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751aabba0 0x561751abbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751927df0 0x561751aa56c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/1845939988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aa5c00 0x561751aaeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1845939988 >> 172.21.15.50:0/1845939988 conn(0x561751966a30 msgr2=0x561751aaa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1845939988 shutdown_connections 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.829+0000 7f1bbd66db80 1 -- 172.21.15.50:0/1845939988 wait complete. 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.830+0000 7f1bbd66db80 1 -- start start 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.830+0000 7f1bbd66db80 1 Processor -- start 2024-05-10T21:08:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.830+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 0x561751aa47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.830+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 0x561751aa4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bbd66db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561751ac0b40 con 0x561751aa5c00 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561751ac0470 con 0x561751927df0 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bbd66db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561751ac1210 con 0x561751aabba0 2024-05-10T21:08:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bbd66db80 10 client.0 _ll_register_callbacks cb 0x56175195b330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 0x561751aa4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 0x561751aa47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1ba7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 0x561751aa47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:42926/0 (socket says 172.21.15.50:42926) 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1ba7e00700 1 -- 172.21.15.50:0/4024867809 learned_addr learned my addr 172.21.15.50:0/4024867809 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1bacc00700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 -- 172.21.15.50:0/4024867809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 msgr2=0x561751aa5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 -- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 msgr2=0x561751aa47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 0x561751aa47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.831+0000 7f1b9f400700 1 -- 172.21.15.50:0/4024867809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561751ac8480 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1b9f400700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 0x561751aa4d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b900aa5e0 tx=0x7f1b90090df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e341560d9fef83a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1b96000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a9fa0 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1b900afb90 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9802ce50 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b900aa160 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1bbd66db80 10 client.14916 fetch_fsmap learned FSMap version 24 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.832+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561751ac8480 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9802ce50 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1b900050f0 con 0x561751aa5c00 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 10 client.14916 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 10 client.14916 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 20 client.14916 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:07.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751b2cfa0 0x561751b2e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:07.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 4 client.14916 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:08:07.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "status"}) -- 0x5617519d53b0 con 0x561751b2cfa0 2024-05-10T21:08:07.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.833+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751b2cfa0 0x561751b2e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.834+0000 7f1ba7e00700 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751b2cfa0 0x561751b2e410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4402a48182f31e4b server_cookie=b7f690da5968788d in_seq=0 out_seq=0 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.834+0000 7f1b96000700 10 client.14916 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.834+0000 7f1b96000700 1 -- 172.21.15.50:0/4024867809 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5617519d53b0 con 0x561751b2cfa0 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.834+0000 7f1b96000700 10 client.14916 handle_command_reply: tid=0 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.834+0000 7f1bbd66db80 1 client.14916 shutdown 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 msgr2=0x561751aa4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 0x561751aa4d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b900aa5e0 tx=0x7f1b90090df0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 shutdown_connections 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561751b2cfa0 0x561751b2e410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561751927df0 0x561751aa47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561751aabba0 0x561751aa5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 --2- 172.21.15.50:0/4024867809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561751aa5c00 0x561751aa4d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 >> 172.21.15.50:0/4024867809 conn(0x561751966a30 msgr2=0x561751aaac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1b96000700 0 client.14916 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 shutdown_connections 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 1 -- 172.21.15.50:0/4024867809 wait complete. 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.835+0000 7f1bbd66db80 20 client.14916 trim_cache size 0 max 16384 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-10T21:08:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24539, 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 220.89144200999999, 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 24, 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 80, 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 243.47218692499999 2024-05-10T21:08:07.839 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 msgr2=0x7f1b8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 0x7f1b8c0b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x561751aa69e0 tx=0x7f1ba0004750 comp rx=0 tx=0).stop 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 msgr2=0x7f1ba81a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000c00 tx=0x7f1b900057c0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 shutdown_connections 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1b8c0b2900 0x7f1b8c0b4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8115370 0x7f1ba81a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba8075b60 0x7f1ba81a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 --2- 172.21.15.50:0/2603527433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba80764b0 0x7f1ba81a19f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 >> 172.21.15.50:0/2603527433 conn(0x7f1ba8100040 msgr2=0x7f1ba8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 shutdown_connections 2024-05-10T21:08:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:07.836+0000 7f1baec00700 1 -- 172.21.15.50:0/2603527433 wait complete. 2024-05-10T21:08:08.741 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T21:08:08.752 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T21:08:08.752 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T21:08:08.753 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T21:08:08.753 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T21:08:08.825 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T21:08:08.826 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-10T21:08:08.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:08.826 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-10T21:08:08.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:08.890 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-05-10T21:08:08.966 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:08 smithi050 ceph-mon[29546]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:08.987 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-05-10T21:08:08.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:08:08.992 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-10T21:08:08.992 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T21:08:08.992 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T21:08:08.992 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T21:08:09.182 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:08 smithi142 ceph-mon[34846]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:09.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:08 smithi092 ceph-mon[34415]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:11.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:10 smithi050 ceph-mon[29546]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:11.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:10 smithi092 ceph-mon[34415]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:11.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:10 smithi142 ceph-mon[34846]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:11.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 -- 172.21.15.50:0/1557426911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106580 msgr2=0x7f5ddc106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:11.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 --2- 172.21.15.50:0/1557426911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106580 0x7f5ddc106990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009990 tx=0x7f5dc8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 -- 172.21.15.50:0/1557426911 shutdown_connections 2024-05-10T21:08:11.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 --2- 172.21.15.50:0/1557426911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 0x7f5ddc10c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 --2- 172.21.15.50:0/1557426911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106ed0 0x7f5ddc113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 --2- 172.21.15.50:0/1557426911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106580 0x7f5ddc106990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 -- 172.21.15.50:0/1557426911 >> 172.21.15.50:0/1557426911 conn(0x7f5ddc102080 msgr2=0x7f5ddc1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 -- 172.21.15.50:0/1557426911 shutdown_connections 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.812+0000 7f5de3600700 1 -- 172.21.15.50:0/1557426911 wait complete. 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.814+0000 7f5de3600700 1 Processor -- start 2024-05-10T21:08:11.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.814+0000 7f5de3600700 1 -- start start 2024-05-10T21:08:11.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106ed0 0x7f5ddc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 0x7f5ddc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1166a0 con 0x7f5ddc106580 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc116520 con 0x7f5ddc10c5c0 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de3600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc116820 con 0x7f5ddc106ed0 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de0c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de0c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53206/0 (socket says 172.21.15.50:53206) 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de0c00700 1 -- 172.21.15.50:0/2369412627 learned_addr learned my addr 172.21.15.50:0/2369412627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5ddbe00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106ed0 0x7f5ddc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:11.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.815+0000 7f5de1600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 0x7f5ddc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:11.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 -- 172.21.15.50:0/2369412627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106ed0 msgr2=0x7f5ddc19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:11.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106ed0 0x7f5ddc19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 -- 172.21.15.50:0/2369412627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 msgr2=0x7f5ddc1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:11.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 0x7f5ddc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:11.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 -- 172.21.15.50:0/2369412627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a1ec0 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5de0c00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e970 tx=0x7f5dd000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.816+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004a30 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.817+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a21b0 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.817+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd000cc90 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.817+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005610 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.817+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a9ad0 con 0x7f5ddc106580 2024-05-10T21:08:11.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.818+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc0512c0 con 0x7f5ddc106580 2024-05-10T21:08:11.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.822+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5dd000c7e0 con 0x7f5ddc106580 2024-05-10T21:08:11.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.824+0000 7f5dd9600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 0x7f5dc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:11.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.824+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5dd00e2650 con 0x7f5ddc106580 2024-05-10T21:08:11.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.824+0000 7f5ddbe00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 0x7f5dc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:11.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.825+0000 7f5ddbe00700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 0x7f5dc40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:11.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:11.827+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5dd0054df0 con 0x7f5ddc106580 2024-05-10T21:08:12.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.124+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ddc19e350 con 0x7f5ddc106580 2024-05-10T21:08:12.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.125+0000 7f5dd9600700 1 -- 172.21.15.50:0/2369412627 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f5dd0091070 con 0x7f5ddc106580 2024-05-10T21:08:12.126 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.128+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 msgr2=0x7f5dc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.128+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 0x7f5dc40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8005960 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.128+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 msgr2=0x7f5ddc19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.128+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e970 tx=0x7f5dd000ec80 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 shutdown_connections 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5dc40b2880 0x7f5dc40b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ddc10c5c0 0x7f5ddc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ddc106580 0x7f5ddc19cff0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 --2- 172.21.15.50:0/2369412627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ddc106ed0 0x7f5ddc19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 >> 172.21.15.50:0/2369412627 conn(0x7f5ddc102080 msgr2=0x7f5ddc104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 shutdown_connections 2024-05-10T21:08:12.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:12.129+0000 7f5de3600700 1 -- 172.21.15.50:0/2369412627 wait complete. 2024-05-10T21:08:13.018 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T21:08:13.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T21:08:13.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-05-10T21:08:13.018 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T21:08:13.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T21:08:13.019 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T21:08:13.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T21:08:13.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T21:08:13.019 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T21:08:13.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:13.019 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-05-10T21:08:13.070 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-05-10T21:08:13.071 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41776sec preferred_lft 41776sec 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-05-10T21:08:13.072 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-05-10T21:08:13.073 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T21:08:13.073 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-10T21:08:13.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:12 smithi050 ceph-mon[29546]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:13.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:12 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2369412627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T21:08:13.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:13.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:12 smithi092 ceph-mon[34415]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:13.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:12 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2369412627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T21:08:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:12 smithi142 ceph-mon[34846]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:08:13.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:12 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2369412627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T21:08:13.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:13.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:13.541 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T21:08:13.605 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-05-10T21:08:13.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:13.609 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-05-10T21:08:13.666 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-05-10T21:08:13.667 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T21:08:13.667 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T21:08:13.667 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T21:08:13.667 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T21:08:13.667 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T21:08:13.667 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-10T21:08:13.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T21:08:13.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T21:08:13.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T21:08:13.668 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-10T21:08:13.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:14.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:14.437 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-10T21:08:14.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:14.491 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-05-10T21:08:14.541 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T21:08:14.541 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-10T21:08:14.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T21:08:14.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T21:08:14.541 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-10T21:08:14.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:14.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:14.740 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T21:08:14.740 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T21:08:14.740 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-10T21:08:14.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T21:08:14.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T21:08:14.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T21:08:14.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T21:08:14.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T21:08:14.741 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-10T21:08:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:14.939 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:14 smithi050 ceph-mon[29546]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:15.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:15.062 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T21:08:15.062 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-10T21:08:15.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-05-10T21:08:15.062 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T21:08:15.062 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-10T21:08:15.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:15.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:15.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T21:08:15.276 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T21:08:15.322 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T21:08:15.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T21:08:15.323 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T21:08:15.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:14 smithi092 ceph-mon[34415]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:15.375 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-10T21:08:15.375 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-10T21:08:15.375 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f 2024-05-10T21:08:15.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:15.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-10T21:08:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:14 smithi142 ceph-mon[34846]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:08:15.619 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-10T21:08:15.619 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-10T21:08:16.363 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-10T21:08:16.363 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-10T21:08:16.363 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.92:3300,172.21.15.142:3300" 2024-05-10T21:08:16.364 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with new device syntax: 0@55ba4db6-0f0f-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f 2024-05-10T21:08:16.364 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.50:3300/172.21.15.92:3300/172.21.15.142:3300" will pass to kernel 2024-05-10T21:08:16.371 INFO:teuthology.orchestra.run.smithi050.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T21:08:16.371 INFO:teuthology.orchestra.run.smithi050.stdout: You just mounted a file system that supports labels which does not 2024-05-10T21:08:16.371 INFO:teuthology.orchestra.run.smithi050.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T21:08:16.371 INFO:teuthology.orchestra.run.smithi050.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T21:08:16.371 INFO:teuthology.orchestra.run.smithi050.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T21:08:16.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T21:08:16.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:08:16.378 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T21:08:16.473 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T21:08:16.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T21:08:16.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:16.601 INFO:teuthology.orchestra.run.smithi050.stdout:client14925 2024-05-10T21:08:16.602 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:08:16.602 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T21:08:16.658 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T21:08:16.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T21:08:16.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:16.784 INFO:teuthology.orchestra.run.smithi050.stdout:client14925 2024-05-10T21:08:16.784 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T21:08:16.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T21:08:16.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:16.908 INFO:teuthology.orchestra.run.smithi050.stdout:client14925 2024-05-10T21:08:16.908 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968.client14925/status') 2024-05-10T21:08:17.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T21:08:17.048 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.14925 (3)192.168.0.1:0/872562895 2024-05-10T21:08:17.048 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-05-10T21:08:17.048 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-05-10T21:08:17.048 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-05-10T21:08:17.048 INFO:teuthology.orchestra.run.smithi050.stderr:69 bytes copied, 8.3646e-05 s, 825 kB/s 2024-05-10T21:08:17.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T21:08:17.061 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T21:08:17.133 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T21:08:17.133 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:08:17.133 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-10T21:08:17.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:16 smithi050 ceph-mon[29546]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:17.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:16 smithi092 ceph-mon[34415]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:17.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:16 smithi142 ceph-mon[34846]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 -- 172.21.15.50:0/2047568553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8104dd0 msgr2=0x7f4cd81051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 --2- 172.21.15.50:0/2047568553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8104dd0 0x7f4cd81051e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009900 tx=0x7f4cc402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 -- 172.21.15.50:0/2047568553 shutdown_connections 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 --2- 172.21.15.50:0/2047568553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8104dd0 0x7f4cd81051e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 --2- 172.21.15.50:0/2047568553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd810bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 --2- 172.21.15.50:0/2047568553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8103b90 0x7f4cd8104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 -- 172.21.15.50:0/2047568553 >> 172.21.15.50:0/2047568553 conn(0x7f4cd80fed80 msgr2=0x7f4cd8101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 -- 172.21.15.50:0/2047568553 shutdown_connections 2024-05-10T21:08:17.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.423+0000 7f4cdde00640 1 -- 172.21.15.50:0/2047568553 wait complete. 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 Processor -- start 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 -- start start 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 0x7f4cd819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 0x7f4cd819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd81a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810f620 con 0x7f4cd8109ad0 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810f4a0 con 0x7f4cd8104dd0 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.425+0000 7f4cdde00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810f7a0 con 0x7f4cd8103b90 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cdca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 0x7f4cd819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 0x7f4cd819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cdca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 0x7f4cd819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54614/0 (socket says 172.21.15.50:54614) 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cdca00640 1 -- 172.21.15.50:0/3639495868 learned_addr learned my addr 172.21.15.50:0/3639495868 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cdd400640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd81a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 -- 172.21.15.50:0/3639495868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 msgr2=0x7f4cd819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 0x7f4cd819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 -- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 msgr2=0x7f4cd81a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd81a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cd7e00640 1 -- 172.21.15.50:0/3639495868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd81a3f10 con 0x7f4cd8104dd0 2024-05-10T21:08:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.426+0000 7f4cdd400640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd81a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:08:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cd7e00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 0x7f4cd819c910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4cc402a9e0 tx=0x7f4cc4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc403f070 con 0x7f4cd8104dd0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc4031c50 con 0x7f4cd8104dd0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81a41a0 con 0x7f4cd8104dd0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4035030 con 0x7f4cd8104dd0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.427+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a4660 con 0x7f4cd8104dd0 2024-05-10T21:08:17.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.428+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd81a4980 con 0x7f4cd8104dd0 2024-05-10T21:08:17.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.431+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4cc4042070 con 0x7f4cd8104dd0 2024-05-10T21:08:17.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.432+0000 7f4cd5600640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 0x7f4cc00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:17.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.433+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f4cc4103390 con 0x7f4cd8104dd0 2024-05-10T21:08:17.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.433+0000 7f4cdca00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 0x7f4cc00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:17.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.434+0000 7f4cdca00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 0x7f4cc00b4cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009c30 tx=0x7f4ccc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:17.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.435+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cc40cb080 con 0x7f4cd8104dd0 2024-05-10T21:08:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.693+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4cd81a4b50 con 0x7f4cd8104dd0 2024-05-10T21:08:17.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.922+0000 7f4cd5600640 1 -- 172.21.15.50:0/3639495868 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f4cc40ca980 con 0x7f4cd8104dd0 2024-05-10T21:08:17.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.925+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 msgr2=0x7f4cc00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.925+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 0x7f4cc00b4cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009c30 tx=0x7f4ccc005f90 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.925+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 msgr2=0x7f4cd819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.925+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 0x7f4cd819c910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4cc402a9e0 tx=0x7f4cc4003c10 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 shutdown_connections 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4cc00b27e0 0x7f4cc00b4cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cd8109ad0 0x7f4cd81a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd8104dd0 0x7f4cd819c910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 --2- 172.21.15.50:0/3639495868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd8103b90 0x7f4cd819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 >> 172.21.15.50:0/3639495868 conn(0x7f4cd80fed80 msgr2=0x7f4cd80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 shutdown_connections 2024-05-10T21:08:17.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:17.926+0000 7f4cdde00640 1 -- 172.21.15.50:0/3639495868 wait complete. 2024-05-10T21:08:17.964 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T21:08:17.979 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T21:08:17.990 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T21:08:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.294+0000 7f19be400640 1 -- 172.21.15.50:0/331907086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 msgr2=0x7f19b8113420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.294+0000 7f19be400640 1 --2- 172.21.15.50:0/331907086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b8113420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009900 tx=0x7f19b002a400 comp rx=0 tx=0).stop 2024-05-10T21:08:18.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 -- 172.21.15.50:0/331907086 shutdown_connections 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 --2- 172.21.15.50:0/331907086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b8113420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 --2- 172.21.15.50:0/331907086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810e3f0 0x7f19b8110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 --2- 172.21.15.50:0/331907086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810ba90 0x7f19b810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 -- 172.21.15.50:0/331907086 >> 172.21.15.50:0/331907086 conn(0x7f19b8070a50 msgr2=0x7f19b8072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 -- 172.21.15.50:0/331907086 shutdown_connections 2024-05-10T21:08:18.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.295+0000 7f19be400640 1 -- 172.21.15.50:0/331907086 wait complete. 2024-05-10T21:08:18.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.296+0000 7f19be400640 1 Processor -- start 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.296+0000 7f19be400640 1 -- start start 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810ba90 0x7f19b810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b810c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8116e70 con 0x7f19b8110f10 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8116cf0 con 0x7f19b810ba90 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19be400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8116ff0 con 0x7f19b810e3f0 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b810c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54640/0 (socket says 172.21.15.50:54640) 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 -- 172.21.15.50:0/4032583439 learned_addr learned my addr 172.21.15.50:0/4032583439 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:18.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b7400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810ba90 0x7f19b810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 -- 172.21.15.50:0/4032583439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810ba90 msgr2=0x7f19b810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810ba90 0x7f19b810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 -- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 msgr2=0x7f19b810c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b810c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.297+0000 7f19b6a00640 1 -- 172.21.15.50:0/4032583439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b810cc30 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19b6a00640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19a000e9c0 tx=0x7f19a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000cd80 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b81acbd0 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a00054d0 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0010430 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.298+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b81ad0c0 con 0x7f19b810e3f0 2024-05-10T21:08:18.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.299+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b81ad350 con 0x7f19b810e3f0 2024-05-10T21:08:18.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.302+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19a0004fb0 con 0x7f19b810e3f0 2024-05-10T21:08:18.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.304+0000 7f19afe00640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 0x7f199c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.304+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f19a00e2850 con 0x7f19b810e3f0 2024-05-10T21:08:18.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.304+0000 7f19b7400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 0x7f199c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.305+0000 7f19b7400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 0x7f199c0b4d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19b810a1a0 tx=0x7f19a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:18.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.306+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19a0054df0 con 0x7f19b810e3f0 2024-05-10T21:08:18.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:17 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3639495868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:17 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:17 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3639495868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:17 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:17 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3639495868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:17 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T21:08:18.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.520+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19b81ad520 con 0x7f19b810e3f0 2024-05-10T21:08:18.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.522+0000 7f19afe00640 1 -- 172.21.15.50:0/4032583439 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7f19a00aa540 con 0x7f19b810e3f0 2024-05-10T21:08:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:18.522 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:08:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 msgr2=0x7f199c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 0x7f199c0b4d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19b810a1a0 tx=0x7f19a8009500 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 msgr2=0x7f19b810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19a000e9c0 tx=0x7f19a000ee80 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 shutdown_connections 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f199c0b27f0 0x7f199c0b4d00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b810e3f0 0x7f19b810bfc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b8110f10 0x7f19b810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 --2- 172.21.15.50:0/4032583439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b810ba90 0x7f19b810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 >> 172.21.15.50:0/4032583439 conn(0x7f19b8070a50 msgr2=0x7f19b810ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 shutdown_connections 2024-05-10T21:08:18.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.525+0000 7f19be400640 1 -- 172.21.15.50:0/4032583439 wait complete. 2024-05-10T21:08:18.554 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T20:58:45.564017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T21:04:18.047634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T21:04:22.866396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T21:08:18.554 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T21:08:18.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 -- 172.21.15.50:0/1774106752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410bca0 msgr2=0x7f271410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 --2- 172.21.15.50:0/1774106752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410bca0 0x7f271410c0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009900 tx=0x7f26fc02a410 comp rx=0 tx=0).stop 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 -- 172.21.15.50:0/1774106752 shutdown_connections 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 --2- 172.21.15.50:0/1774106752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 0x7f2714112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 --2- 172.21.15.50:0/1774106752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410bca0 0x7f271410c0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 --2- 172.21.15.50:0/1774106752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410ae30 0x7f271410b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 -- 172.21.15.50:0/1774106752 >> 172.21.15.50:0/1774106752 conn(0x7f2714070a70 msgr2=0x7f2714072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 -- 172.21.15.50:0/1774106752 shutdown_connections 2024-05-10T21:08:18.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.849+0000 7f2718c00640 1 -- 172.21.15.50:0/1774106752 wait complete. 2024-05-10T21:08:18.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.850+0000 7f2718c00640 1 Processor -- start 2024-05-10T21:08:18.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 -- start start 2024-05-10T21:08:18.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 0x7f27141a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 0x7f27141a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 0x7f27141aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2714118030 con 0x7f27141109a0 2024-05-10T21:08:18.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2714117eb0 con 0x7f271410ae30 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2718c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27141181b0 con 0x7f271410bca0 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.851+0000 7f2711400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 0x7f27141a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 0x7f27141a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2712800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 0x7f27141aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 0x7f27141a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54656/0 (socket says 172.21.15.50:54656) 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711400640 1 -- 172.21.15.50:0/3000658915 learned_addr learned my addr 172.21.15.50:0/3000658915 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 -- 172.21.15.50:0/3000658915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 msgr2=0x7f27141a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 0x7f27141a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 -- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 msgr2=0x7f27141aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 0x7f27141aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:18.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 -- 172.21.15.50:0/3000658915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141aa740 con 0x7f271410ae30 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.852+0000 7f2711e00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 0x7f27141a2c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2704005fd0 tx=0x7f270400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.853+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704004b20 con 0x7f271410ae30 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.853+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2704004cc0 con 0x7f271410ae30 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.853+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704010690 con 0x7f271410ae30 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.853+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141aaa30 con 0x7f271410ae30 2024-05-10T21:08:18.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.853+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141aaf80 con 0x7f271410ae30 2024-05-10T21:08:18.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.854+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271410f7b0 con 0x7f271410ae30 2024-05-10T21:08:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.857+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2704005020 con 0x7f271410ae30 2024-05-10T21:08:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.858+0000 7f270aa00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 0x7f26f80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.858+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f27040e2930 con 0x7f271410ae30 2024-05-10T21:08:18.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.859+0000 7f2711400640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 0x7f26f80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.859+0000 7f2711400640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 0x7f26f80b4d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005f70 tx=0x7f26fc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:18.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:18.860+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27040aa620 con 0x7f271410ae30 2024-05-10T21:08:19.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.078+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f271402ddc0 con 0x7f271410ae30 2024-05-10T21:08:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.079+0000 7f270aa00640 1 -- 172.21.15.50:0/3000658915 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f27040a9f20 con 0x7f271410ae30 2024-05-10T21:08:19.080 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-05-10T21:08:19.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.082+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 msgr2=0x7f26f80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.082+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 0x7f26f80b4d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005f70 tx=0x7f26fc0314a0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.082+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 msgr2=0x7f27141a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.082+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 0x7f27141a2c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2704005fd0 tx=0x7f270400ef90 comp rx=0 tx=0).stop 2024-05-10T21:08:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 shutdown_connections 2024-05-10T21:08:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f26f80b2870 0x7f26f80b4d80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f271410bca0 0x7f27141a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27141109a0 0x7f27141aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 --2- 172.21.15.50:0/3000658915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f271410ae30 0x7f27141a2c00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 >> 172.21.15.50:0/3000658915 conn(0x7f2714070a70 msgr2=0x7f271410c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 shutdown_connections 2024-05-10T21:08:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.083+0000 7f2718c00640 1 -- 172.21.15.50:0/3000658915 wait complete. 2024-05-10T21:08:19.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T21:08:19.184 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:18 smithi050 ceph-mon[29546]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:08:19.184 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:18 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T21:08:19.184 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:18 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:08:19.184 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:18 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4032583439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:19.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:18 smithi092 ceph-mon[34415]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:08:19.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:18 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T21:08:19.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:18 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:08:19.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:18 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4032583439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:19.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:18 smithi142 ceph-mon[34846]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:08:19.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:18 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T21:08:19.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:18 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:08:19.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:18 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4032583439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:19.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 -- 172.21.15.50:0/3613566931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 msgr2=0x7f8454105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 --2- 172.21.15.50:0/3613566931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454105270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8444009900 tx=0x7f844402a410 comp rx=0 tx=0).stop 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 -- 172.21.15.50:0/3613566931 shutdown_connections 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 --2- 172.21.15.50:0/3613566931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454109b60 0x7f845410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 --2- 172.21.15.50:0/3613566931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454105270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 --2- 172.21.15.50:0/3613566931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454103c20 0x7f8454104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 -- 172.21.15.50:0/3613566931 >> 172.21.15.50:0/3613566931 conn(0x7f84540fee10 msgr2=0x7f84541012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 -- 172.21.15.50:0/3613566931 shutdown_connections 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.405+0000 7f845ba00640 1 -- 172.21.15.50:0/3613566931 wait complete. 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.406+0000 7f845ba00640 1 Processor -- start 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.406+0000 7f845ba00640 1 -- start start 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454103c20 0x7f8454077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454109b60 0x7f8454076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f845410f7a0 con 0x7f8454104e60 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f845410f620 con 0x7f8454109b60 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f845ba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f845410f920 con 0x7f8454103c20 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f8453e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f8459000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454103c20 0x7f8454077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f8453e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46976/0 (socket says 172.21.15.50:46976) 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f8453e00640 1 -- 172.21.15.50:0/1373603268 learned_addr learned my addr 172.21.15.50:0/1373603268 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:19.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.407+0000 7f8459a00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454109b60 0x7f8454076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 -- 172.21.15.50:0/1373603268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454103c20 msgr2=0x7f8454077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454103c20 0x7f8454077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 -- 172.21.15.50:0/1373603268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454109b60 msgr2=0x7f8454076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454109b60 0x7f8454076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 -- 172.21.15.50:0/1373603268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8454076da0 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8453e00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8444031440 tx=0x7f8444003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844403f070 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.408+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8444031c50 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.409+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84541a43b0 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.409+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444035030 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.409+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a47b0 con 0x7f8454104e60 2024-05-10T21:08:19.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.410+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845406a290 con 0x7f8454104e60 2024-05-10T21:08:19.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.412+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8444042070 con 0x7f8454104e60 2024-05-10T21:08:19.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.414+0000 7f8451600640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 0x7f843c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.414+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f84441031f0 con 0x7f8454104e60 2024-05-10T21:08:19.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.414+0000 7f8459000640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 0x7f843c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.415+0000 7f8459000640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 0x7f843c0b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8448005fd0 tx=0x7f844800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:19.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.417+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84440ca890 con 0x7f8454104e60 2024-05-10T21:08:19.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.629+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f845402dfc0 con 0x7f8454104e60 2024-05-10T21:08:19.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.630+0000 7f8451600640 1 -- 172.21.15.50:0/1373603268 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f84440ceb90 con 0x7f8454104e60 2024-05-10T21:08:19.630 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 msgr2=0x7f843c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 0x7f843c0b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8448005fd0 tx=0x7f844800b410 comp rx=0 tx=0).stop 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 msgr2=0x7f8454076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8444031440 tx=0x7f8444003c10 comp rx=0 tx=0).stop 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 shutdown_connections 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f843c0b27f0 0x7f843c0b4d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8454109b60 0x7f8454076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454104e60 0x7f8454076130 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 --2- 172.21.15.50:0/1373603268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8454103c20 0x7f8454077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 >> 172.21.15.50:0/1373603268 conn(0x7f84540fee10 msgr2=0x7f8454105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 shutdown_connections 2024-05-10T21:08:19.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.633+0000 7f845ba00640 1 -- 172.21.15.50:0/1373603268 wait complete. 2024-05-10T21:08:19.661 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T21:08:19.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 -- 172.21.15.50:0/1205636642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 msgr2=0x7fa870104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 --2- 172.21.15.50:0/1205636642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870104920 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa86002a410 comp rx=0 tx=0).stop 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 -- 172.21.15.50:0/1205636642 shutdown_connections 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 --2- 172.21.15.50:0/1205636642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 0x7fa870105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 --2- 172.21.15.50:0/1205636642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 0x7fa87010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 --2- 172.21.15.50:0/1205636642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870104920 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 -- 172.21.15.50:0/1205636642 >> 172.21.15.50:0/1205636642 conn(0x7fa8700fee10 msgr2=0x7fa8701012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 -- 172.21.15.50:0/1205636642 shutdown_connections 2024-05-10T21:08:19.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.963+0000 7fa875a00640 1 -- 172.21.15.50:0/1205636642 wait complete. 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.964+0000 7fa875a00640 1 Processor -- start 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.964+0000 7fa875a00640 1 -- start start 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 0x7fa870076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 0x7fa870076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa87010f750 con 0x7fa870103c20 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa87010f5d0 con 0x7fa870109b60 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa875a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa87010f8d0 con 0x7fa870104e60 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47006/0 (socket says 172.21.15.50:47006) 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 -- 172.21.15.50:0/1755218467 learned_addr learned my addr 172.21.15.50:0/1755218467 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa866000640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 0x7fa870076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86f400640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 0x7fa870076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 -- 172.21.15.50:0/1755218467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 msgr2=0x7fa870076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 0x7fa870076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 -- 172.21.15.50:0/1755218467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 msgr2=0x7fa870076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 0x7fa870076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.965+0000 7fa86ea00640 1 -- 172.21.15.50:0/1755218467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019e1f0 con 0x7fa870103c20 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa86ea00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa860031440 tx=0x7fa860003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:19.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86003f070 con 0x7fa870103c20 2024-05-10T21:08:19.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa860031c50 con 0x7fa870103c20 2024-05-10T21:08:19.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860035030 con 0x7fa870103c20 2024-05-10T21:08:19.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa870076c70 con 0x7fa870103c20 2024-05-10T21:08:19.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.966+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a8c10 con 0x7fa870103c20 2024-05-10T21:08:19.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.968+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa870108970 con 0x7fa870103c20 2024-05-10T21:08:19.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.972+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa860042070 con 0x7fa870103c20 2024-05-10T21:08:19.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.974+0000 7fa867e00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 0x7fa8540b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:19.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.974+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa860103160 con 0x7fa870103c20 2024-05-10T21:08:19.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.974+0000 7fa866000640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 0x7fa8540b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:19.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.975+0000 7fa866000640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 0x7fa8540b4d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa870077430 tx=0x7fa858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:19.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:19.976+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8600ca800 con 0x7fa870103c20 2024-05-10T21:08:20.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.191+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8700638d0 con 0x7fa870103c20 2024-05-10T21:08:20.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.191+0000 7fa867e00640 1 -- 172.21.15.50:0/1755218467 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fa8600ceb00 con 0x7fa870103c20 2024-05-10T21:08:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-05-10T21:08:20.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 msgr2=0x7fa8540b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:20.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 0x7fa8540b4d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa870077430 tx=0x7fa858005f90 comp rx=0 tx=0).stop 2024-05-10T21:08:20.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 msgr2=0x7fa870077a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:20.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa860031440 tx=0x7fa860003c10 comp rx=0 tx=0).stop 2024-05-10T21:08:20.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 shutdown_connections 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8540b27f0 0x7fa8540b4d00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa870104e60 0x7fa870076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa870109b60 0x7fa870076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 --2- 172.21.15.50:0/1755218467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa870103c20 0x7fa870077a90 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa860031440 tx=0x7fa860003c10 comp rx=0 tx=0).stop 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 >> 172.21.15.50:0/1755218467 conn(0x7fa8700fee10 msgr2=0x7fa870105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.194+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 shutdown_connections 2024-05-10T21:08:20.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:20.195+0000 7fa875a00640 1 -- 172.21.15.50:0/1755218467 wait complete. 2024-05-10T21:08:20.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3000658915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.217 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:19 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1373603268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.220 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T21:08:20.220 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:08:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3000658915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.270 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:19 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1373603268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3000658915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:19 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1373603268' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:08:20.692 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:21.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:20 smithi050 ceph-mon[29546]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:08:21.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1755218467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:08:21.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:20 smithi092 ceph-mon[34415]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:08:21.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1755218467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:08:21.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:20 smithi142 ceph-mon[34846]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:08:21.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1755218467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:08:22.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:21 smithi092 ceph-mon[34415]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:08:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:21 smithi142 ceph-mon[34846]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:08:22.382 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:21 smithi050 ceph-mon[29546]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 -- 172.21.15.50:0/186542185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f0106620 msgr2=0x7fd8f0106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 --2- 172.21.15.50:0/186542185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f0106620 0x7fd8f0106a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00099c0 tx=0x7fd8e0009cd0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 -- 172.21.15.50:0/186542185 shutdown_connections 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 --2- 172.21.15.50:0/186542185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 0x7fd8f0113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 --2- 172.21.15.50:0/186542185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f0106620 0x7fd8f0106a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 --2- 172.21.15.50:0/186542185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f010af50 0x7fd8f010b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.009+0000 7fd8efe00700 1 -- 172.21.15.50:0/186542185 >> 172.21.15.50:0/186542185 conn(0x7fd8f0102030 msgr2=0x7fd8f0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.010+0000 7fd8efe00700 1 -- 172.21.15.50:0/186542185 shutdown_connections 2024-05-10T21:08:23.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.010+0000 7fd8efe00700 1 -- 172.21.15.50:0/186542185 wait complete. 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.011+0000 7fd8efe00700 1 Processor -- start 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.011+0000 7fd8efe00700 1 -- start start 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.011+0000 7fd8efe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.011+0000 7fd8efe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 0x7fd8f019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.011+0000 7fd8efe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 0x7fd8f01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8efe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01164d0 con 0x7fd8f010af50 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8efe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0116350 con 0x7fd8f0106620 2024-05-10T21:08:23.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8efe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0116650 con 0x7fd8f010b8a0 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49786/0 (socket says 172.21.15.50:49786) 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8eea00700 1 -- 172.21.15.50:0/3021611375 learned_addr learned my addr 172.21.15.50:0/3021611375 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 0x7fd8f019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ef400700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 0x7fd8f01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 -- 172.21.15.50:0/3021611375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 msgr2=0x7fd8f01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 0x7fd8f01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 -- 172.21.15.50:0/3021611375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 msgr2=0x7fd8f019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.012+0000 7fd8ee000700 1 -- 172.21.15.50:0/3021611375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f01a1d90 con 0x7fd8f010af50 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8eea00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8ee000700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 0x7fd8f019d460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00e9c0 tx=0x7fd8dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc00c9c0 con 0x7fd8f010af50 2024-05-10T21:08:23.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8dc00cb60 con 0x7fd8f010af50 2024-05-10T21:08:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc018680 con 0x7fd8f010af50 2024-05-10T21:08:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f01a2080 con 0x7fd8f010af50 2024-05-10T21:08:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.013+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01a99a0 con 0x7fd8f010af50 2024-05-10T21:08:23.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.015+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f00512c0 con 0x7fd8f010af50 2024-05-10T21:08:23.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.017+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd8dc010430 con 0x7fd8f010af50 2024-05-10T21:08:23.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.019+0000 7fd8e7400700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 0x7fd8d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:23.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.019+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fd8dc0e23b0 con 0x7fd8f010af50 2024-05-10T21:08:23.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.019+0000 7fd8eea00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 0x7fd8d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:23.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.020+0000 7fd8eea00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 0x7fd8d40b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009990 tx=0x7fd8e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:23.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.024+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8dc0aa0a0 con 0x7fd8f010af50 2024-05-10T21:08:23.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.312+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8f002e5f0 con 0x7fd8f010af50 2024-05-10T21:08:23.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.314+0000 7fd8e7400700 1 -- 172.21.15.50:0/3021611375 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fd8dc0b2120 con 0x7fd8f010af50 2024-05-10T21:08:23.315 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:23.315 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 msgr2=0x7fd8d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 0x7fd8d40b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009990 tx=0x7fd8e001a040 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 msgr2=0x7fd8f019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 0x7fd8f019d460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00e9c0 tx=0x7fd8dc00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 shutdown_connections 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd8d40b2880 0x7fd8d40b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0106620 0x7fd8f019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8f010b8a0 0x7fd8f01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 --2- 172.21.15.50:0/3021611375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8f010af50 0x7fd8f019d460 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 >> 172.21.15.50:0/3021611375 conn(0x7fd8f0102030 msgr2=0x7fd8f010a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.317+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 shutdown_connections 2024-05-10T21:08:23.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:23.318+0000 7fd8efe00700 1 -- 172.21.15.50:0/3021611375 wait complete. 2024-05-10T21:08:23.319 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:08:23.606 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:23 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3021611375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:23.619 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:23 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3021611375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:23.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:23 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3021611375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:24.117 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T21:08:24.117 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T21:08:24.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 -- 172.21.15.50:0/4201474046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810e600 msgr2=0x7fc53810f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 --2- 172.21.15.50:0/4201474046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810e600 0x7fc53810f300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c02a3f0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 -- 172.21.15.50:0/4201474046 shutdown_connections 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 --2- 172.21.15.50:0/4201474046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc53810f840 0x7fc53810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 --2- 172.21.15.50:0/4201474046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc538114540 0x7fc538116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 --2- 172.21.15.50:0/4201474046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810e600 0x7fc53810f300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.395+0000 7fc53f800640 1 -- 172.21.15.50:0/4201474046 >> 172.21.15.50:0/4201474046 conn(0x7fc538074ab0 msgr2=0x7fc53810bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.396+0000 7fc53f800640 1 -- 172.21.15.50:0/4201474046 shutdown_connections 2024-05-10T21:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.396+0000 7fc53f800640 1 -- 172.21.15.50:0/4201474046 wait complete. 2024-05-10T21:08:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.397+0000 7fc53f800640 1 Processor -- start 2024-05-10T21:08:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 -- start start 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810f840 0x7fc5381a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 0x7fc5381ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc53811a0f0 con 0x7fc53810f840 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc538119f70 con 0x7fc53810e600 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53f800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc53811a270 con 0x7fc538114540 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.398+0000 7fc537e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810f840 0x7fc5381a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54632/0 (socket says 172.21.15.50:54632) 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 -- 172.21.15.50:0/283928909 learned_addr learned my addr 172.21.15.50:0/283928909 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53d800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 0x7fc5381ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 -- 172.21.15.50:0/283928909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 msgr2=0x7fc5381ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 0x7fc5381ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 -- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810f840 msgr2=0x7fc5381a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810f840 0x7fc5381a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 -- 172.21.15.50:0/283928909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381ac880 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53d800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 0x7fc5381ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.399+0000 7fc53ce00640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006010 tx=0x7fc52c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.400+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c03f030 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.400+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381acb10 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.400+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c031910 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.400+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c035070 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.400+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381ad050 con 0x7fc53810e600 2024-05-10T21:08:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.401+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538113350 con 0x7fc53810e600 2024-05-10T21:08:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.403+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc52c042030 con 0x7fc53810e600 2024-05-10T21:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.405+0000 7fc535600640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 0x7fc5200b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.405+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc52c102580 con 0x7fc53810e600 2024-05-10T21:08:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.405+0000 7fc537e00640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 0x7fc5200b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.406+0000 7fc537e00640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 0x7fc5200b4d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc524009bd0 tx=0x7fc524005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.410+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc52c0c9c20 con 0x7fc53810e600 2024-05-10T21:08:24.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.618+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc538063110 con 0x7fc5200b27f0 2024-05-10T21:08:24.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.628+0000 7fc535600640 1 -- 172.21.15.50:0/283928909 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258859 (secure 0 0 0) 0x7fc538063110 con 0x7fc5200b27f0 2024-05-10T21:08:24.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:24 smithi142 ceph-mon[34846]: pgmap v446: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:24.636 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 msgr2=0x7fc5200b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 0x7fc5200b4d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc524009bd0 tx=0x7fc524005f90 comp rx=0 tx=0).stop 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 msgr2=0x7fc5381a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006010 tx=0x7fc52c004270 comp rx=0 tx=0).stop 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 shutdown_connections 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc5200b27f0 0x7fc5200b4d00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc538114540 0x7fc5381ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc53810f840 0x7fc5381a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 --2- 172.21.15.50:0/283928909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53810e600 0x7fc5381a4d50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 >> 172.21.15.50:0/283928909 conn(0x7fc538074ab0 msgr2=0x7fc538110520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:24.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 shutdown_connections 2024-05-10T21:08:24.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.635+0000 7fc53f800640 1 -- 172.21.15.50:0/283928909 wait complete. 2024-05-10T21:08:24.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:24 smithi050 ceph-mon[29546]: pgmap v446: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:24.663 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-10T21:08:23.674957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665196,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1403,"num_read_kb":806,"num_write":306,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":53,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006169"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578816+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:25.578816+0000","last_peered":"2024-05-10T21:06:25.578816+0000","last_clean":"2024-05-10T21:06:25.578816+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:25.578816+0000","last_undegraded":"2024-05-10T21:06:25.578816+0000","last_fullsized":"2024-05-10T21:06:25.578816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580291+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:25.580291+0000","last_peered":"2024-05-10T21:06:25.580291+0000","last_clean":"2024-05-10T21:06:25.580291+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:25.580291+0000","last_undegraded":"2024-05-10T21:06:25.580291+0000","last_fullsized":"2024-05-10T21:06:25.580291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581398+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:25.581398+0000","last_peered":"2024-05-10T21:06:25.581398+0000","last_clean":"2024-05-10T21:06:25.581398+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:25.581398+0000","last_undegraded":"2024-05-10T21:06:25.581398+0000","last_fullsized":"2024-05-10T21:06:25.581398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566835+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566835+0000","last_peered":"2024-05-10T21:06:25.566835+0000","last_clean":"2024-05-10T21:06:25.566835+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566835+0000","last_undegraded":"2024-05-10T21:06:25.566835+0000","last_fullsized":"2024-05-10T21:06:25.566835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.184551+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.184551+0000","last_peered":"2024-05-10T21:04:57.184551+0000","last_clean":"2024-05-10T21:04:57.184551+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.184551+0000","last_undegraded":"2024-05-10T21:04:57.184551+0000","last_fullsized":"2024-05-10T21:04:57.184551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569551+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.569551+0000","last_peered":"2024-05-10T21:06:25.569551+0000","last_clean":"2024-05-10T21:06:25.569551+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.569551+0000","last_undegraded":"2024-05-10T21:06:25.569551+0000","last_fullsized":"2024-05-10T21:06:25.569551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581364+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:25.581364+0000","last_peered":"2024-05-10T21:06:25.581364+0000","last_clean":"2024-05-10T21:06:25.581364+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:25.581364+0000","last_undegraded":"2024-05-10T21:06:25.581364+0000","last_fullsized":"2024-05-10T21:06:25.581364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":250,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:19.087787+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:19.087787+0000","last_peered":"2024-05-10T21:08:19.087787+0000","last_clean":"2024-05-10T21:08:19.087787+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:19.087787+0000","last_undegraded":"2024-05-10T21:08:19.087787+0000","last_fullsized":"2024-05-10T21:08:19.087787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:21.090583+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:21.090583+0000","last_peered":"2024-05-10T21:08:21.090583+0000","last_clean":"2024-05-10T21:08:21.090583+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:21.090583+0000","last_undegraded":"2024-05-10T21:08:21.090583+0000","last_fullsized":"2024-05-10T21:08:21.090583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":510,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.092334+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:22.092334+0000","last_peered":"2024-05-10T21:08:22.092334+0000","last_clean":"2024-05-10T21:08:22.092334+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:22.092334+0000","last_undegraded":"2024-05-10T21:08:22.092334+0000","last_fullsized":"2024-05-10T21:08:22.092334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70142,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":446,"num_read_kb":292,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.237859+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:22.237859+0000","last_peered":"2024-05-10T21:08:22.237859+0000","last_clean":"2024-05-10T21:08:22.237859+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:22.237859+0000","last_undegraded":"2024-05-10T21:08:22.237859+0000","last_fullsized":"2024-05-10T21:08:22.237859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893043+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.893043+0000","last_peered":"2024-05-10T21:04:56.893043+0000","last_clean":"2024-05-10T21:04:56.893043+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.893043+0000","last_undegraded":"2024-05-10T21:04:56.893043+0000","last_fullsized":"2024-05-10T21:04:56.893043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576512+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:25.576512+0000","last_peered":"2024-05-10T21:06:25.576512+0000","last_clean":"2024-05-10T21:06:25.576512+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:25.576512+0000","last_undegraded":"2024-05-10T21:06:25.576512+0000","last_fullsized":"2024-05-10T21:06:25.576512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895673+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895673+0000","last_peered":"2024-05-10T21:04:56.895673+0000","last_clean":"2024-05-10T21:04:56.895673+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895673+0000","last_undegraded":"2024-05-10T21:04:56.895673+0000","last_fullsized":"2024-05-10T21:04:56.895673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578224+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:25.578224+0000","last_peered":"2024-05-10T21:06:25.578224+0000","last_clean":"2024-05-10T21:06:25.578224+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:25.578224+0000","last_undegraded":"2024-05-10T21:06:25.578224+0000","last_fullsized":"2024-05-10T21:06:25.578224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555147+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.555147+0000","last_peered":"2024-05-10T21:06:25.555147+0000","last_clean":"2024-05-10T21:06:25.555147+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.555147+0000","last_undegraded":"2024-05-10T21:06:25.555147+0000","last_fullsized":"2024-05-10T21:06:25.555147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:20.233124+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:20.233124+0000","last_peered":"2024-05-10T21:08:20.233124+0000","last_clean":"2024-05-10T21:08:20.233124+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:20.233124+0000","last_undegraded":"2024-05-10T21:08:20.233124+0000","last_fullsized":"2024-05-10T21:08:20.233124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570231+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:25.570231+0000","last_peered":"2024-05-10T21:06:25.570231+0000","last_clean":"2024-05-10T21:06:25.570231+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:25.570231+0000","last_undegraded":"2024-05-10T21:06:25.570231+0000","last_fullsized":"2024-05-10T21:06:25.570231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331885+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331885+0000","last_peered":"2024-05-10T21:04:57.331885+0000","last_clean":"2024-05-10T21:04:57.331885+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331885+0000","last_undegraded":"2024-05-10T21:04:57.331885+0000","last_fullsized":"2024-05-10T21:04:57.331885+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552134+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:25.552134+0000","last_peered":"2024-05-10T21:06:25.552134+0000","last_clean":"2024-05-10T21:06:25.552134+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:25.552134+0000","last_undegraded":"2024-05-10T21:06:25.552134+0000","last_fullsized":"2024-05-10T21:06:25.552134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891596+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891596+0000","last_peered":"2024-05-10T21:04:56.891596+0000","last_clean":"2024-05-10T21:04:56.891596+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891596+0000","last_undegraded":"2024-05-10T21:04:56.891596+0000","last_fullsized":"2024-05-10T21:04:56.891596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562727+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:25.562727+0000","last_peered":"2024-05-10T21:06:25.562727+0000","last_clean":"2024-05-10T21:06:25.562727+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:25.562727+0000","last_undegraded":"2024-05-10T21:06:25.562727+0000","last_fullsized":"2024-05-10T21:06:25.562727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.918836+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:28.918836+0000","last_peered":"2024-05-10T21:06:28.918836+0000","last_clean":"2024-05-10T21:06:28.918836+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:28.918836+0000","last_undegraded":"2024-05-10T21:06:28.918836+0000","last_fullsized":"2024-05-10T21:06:28.918836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557980+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:25.557980+0000","last_peered":"2024-05-10T21:06:25.557980+0000","last_clean":"2024-05-10T21:06:25.557980+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:25.557980+0000","last_undegraded":"2024-05-10T21:06:25.557980+0000","last_fullsized":"2024-05-10T21:06:25.557980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552796+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:25.552796+0000","last_peered":"2024-05-10T21:06:25.552796+0000","last_clean":"2024-05-10T21:06:25.552796+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:25.552796+0000","last_undegraded":"2024-05-10T21:06:25.552796+0000","last_fullsized":"2024-05-10T21:06:25.552796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569414+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:25.569414+0000","last_peered":"2024-05-10T21:06:25.569414+0000","last_clean":"2024-05-10T21:06:25.569414+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:25.569414+0000","last_undegraded":"2024-05-10T21:06:25.569414+0000","last_fullsized":"2024-05-10T21:06:25.569414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579761+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:25.579761+0000","last_peered":"2024-05-10T21:06:25.579761+0000","last_clean":"2024-05-10T21:06:25.579761+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:25.579761+0000","last_undegraded":"2024-05-10T21:06:25.579761+0000","last_fullsized":"2024-05-10T21:06:25.579761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74594,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1327,"num_read_kb":742,"num_write":181,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037836,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430298,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:24.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T21:08:24.869 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:24 smithi092 ceph-mon[34415]: pgmap v446: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:24.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.995+0000 7f0c16000640 1 -- 172.21.15.50:0/3377117579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1010b9d0 msgr2=0x7f0c1010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.995+0000 7f0c16000640 1 --2- 172.21.15.50:0/3377117579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1010b9d0 0x7f0c1010de10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009900 tx=0x7f0c0802a410 comp rx=0 tx=0).stop 2024-05-10T21:08:24.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 -- 172.21.15.50:0/3377117579 shutdown_connections 2024-05-10T21:08:24.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 --2- 172.21.15.50:0/3377117579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1010b9d0 0x7f0c1010de10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 --2- 172.21.15.50:0/3377117579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c10075a10 0x7f0c1010b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 --2- 172.21.15.50:0/3377117579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c100750c0 0x7f0c100754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 -- 172.21.15.50:0/3377117579 >> 172.21.15.50:0/3377117579 conn(0x7f0c100fec50 msgr2=0x7f0c10101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:24.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 -- 172.21.15.50:0/3377117579 shutdown_connections 2024-05-10T21:08:24.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.996+0000 7f0c16000640 1 -- 172.21.15.50:0/3377117579 wait complete. 2024-05-10T21:08:24.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 Processor -- start 2024-05-10T21:08:24.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 -- start start 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c10075a10 0x7f0c1019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 0x7f0c101a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101117e0 con 0x7f0c1010b9d0 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10111660 con 0x7f0c100750c0 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.997+0000 7f0c16000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10111960 con 0x7f0c10075a10 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54638/0 (socket says 172.21.15.50:54638) 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 -- 172.21.15.50:0/2799334511 learned_addr learned my addr 172.21.15.50:0/2799334511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c15600640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 0x7f0c101a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c07e00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c10075a10 0x7f0c1019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 -- 172.21.15.50:0/2799334511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c10075a10 msgr2=0x7f0c1019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c10075a10 0x7f0c1019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:24.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 -- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 msgr2=0x7f0c101a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 0x7f0c101a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c14c00640 1 -- 172.21.15.50:0/2799334511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c101a3e70 con 0x7f0c100750c0 2024-05-10T21:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.998+0000 7f0c15600640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 0x7f0c101a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:25.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c14c00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ebc0 tx=0x7f0c0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004d20 con 0x7f0c100750c0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c000054d0 con 0x7f0c100750c0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00010720 con 0x7f0c100750c0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101a4160 con 0x7f0c100750c0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:24.999+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a4660 con 0x7f0c100750c0 2024-05-10T21:08:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.000+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1010a2a0 con 0x7f0c100750c0 2024-05-10T21:08:25.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.003+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0c00005020 con 0x7f0c100750c0 2024-05-10T21:08:25.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.004+0000 7f0c0e000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 0x7f0bf40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.005+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0c000e2a40 con 0x7f0c100750c0 2024-05-10T21:08:25.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.005+0000 7f0c07e00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 0x7f0bf40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:25.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.006+0000 7f0c07e00640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 0x7f0bf40b4d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c1019d3d0 tx=0x7f0bf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:25.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.007+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c000aa730 con 0x7f0c100750c0 2024-05-10T21:08:25.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.215+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c10063110 con 0x7f0bf40b2860 2024-05-10T21:08:25.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.237+0000 7f0c0e000640 1 -- 172.21.15.50:0/2799334511 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258859 (secure 0 0 0) 0x7f0c10063110 con 0x7f0bf40b2860 2024-05-10T21:08:25.241 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 msgr2=0x7f0bf40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 0x7f0bf40b4d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c1019d3d0 tx=0x7f0bf8005f90 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 msgr2=0x7f0c1019c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ebc0 tx=0x7f0c0000c6a0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 shutdown_connections 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0bf40b2860 0x7f0bf40b4d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c10075a10 0x7f0c1019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c1010b9d0 0x7f0c101a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 --2- 172.21.15.50:0/2799334511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c100750c0 0x7f0c1019c330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.242+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 >> 172.21.15.50:0/2799334511 conn(0x7f0c100fec50 msgr2=0x7f0c10107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.243+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 shutdown_connections 2024-05-10T21:08:25.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.243+0000 7f0c16000640 1 -- 172.21.15.50:0/2799334511 wait complete. 2024-05-10T21:08:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-10T21:08:23.674957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665196,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1403,"num_read_kb":806,"num_write":306,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":53,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006169"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578816+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:25.578816+0000","last_peered":"2024-05-10T21:06:25.578816+0000","last_clean":"2024-05-10T21:06:25.578816+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:25.578816+0000","last_undegraded":"2024-05-10T21:06:25.578816+0000","last_fullsized":"2024-05-10T21:06:25.578816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580291+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:25.580291+0000","last_peered":"2024-05-10T21:06:25.580291+0000","last_clean":"2024-05-10T21:06:25.580291+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:25.580291+0000","last_undegraded":"2024-05-10T21:06:25.580291+0000","last_fullsized":"2024-05-10T21:06:25.580291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581398+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:25.581398+0000","last_peered":"2024-05-10T21:06:25.581398+0000","last_clean":"2024-05-10T21:06:25.581398+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:25.581398+0000","last_undegraded":"2024-05-10T21:06:25.581398+0000","last_fullsized":"2024-05-10T21:06:25.581398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566835+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566835+0000","last_peered":"2024-05-10T21:06:25.566835+0000","last_clean":"2024-05-10T21:06:25.566835+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566835+0000","last_undegraded":"2024-05-10T21:06:25.566835+0000","last_fullsized":"2024-05-10T21:06:25.566835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.184551+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.184551+0000","last_peered":"2024-05-10T21:04:57.184551+0000","last_clean":"2024-05-10T21:04:57.184551+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.184551+0000","last_undegraded":"2024-05-10T21:04:57.184551+0000","last_fullsized":"2024-05-10T21:04:57.184551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569551+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.569551+0000","last_peered":"2024-05-10T21:06:25.569551+0000","last_clean":"2024-05-10T21:06:25.569551+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.569551+0000","last_undegraded":"2024-05-10T21:06:25.569551+0000","last_fullsized":"2024-05-10T21:06:25.569551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581364+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:25.581364+0000","last_peered":"2024-05-10T21:06:25.581364+0000","last_clean":"2024-05-10T21:06:25.581364+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:25.581364+0000","last_undegraded":"2024-05-10T21:06:25.581364+0000","last_fullsized":"2024-05-10T21:06:25.581364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":250,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:19.087787+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:19.087787+0000","last_peered":"2024-05-10T21:08:19.087787+0000","last_clean":"2024-05-10T21:08:19.087787+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:19.087787+0000","last_undegraded":"2024-05-10T21:08:19.087787+0000","last_fullsized":"2024-05-10T21:08:19.087787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:21.090583+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:21.090583+0000","last_peered":"2024-05-10T21:08:21.090583+0000","last_clean":"2024-05-10T21:08:21.090583+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:21.090583+0000","last_undegraded":"2024-05-10T21:08:21.090583+0000","last_fullsized":"2024-05-10T21:08:21.090583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":510,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.092334+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:22.092334+0000","last_peered":"2024-05-10T21:08:22.092334+0000","last_clean":"2024-05-10T21:08:22.092334+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:22.092334+0000","last_undegraded":"2024-05-10T21:08:22.092334+0000","last_fullsized":"2024-05-10T21:08:22.092334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70142,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":446,"num_read_kb":292,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.237859+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:22.237859+0000","last_peered":"2024-05-10T21:08:22.237859+0000","last_clean":"2024-05-10T21:08:22.237859+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:22.237859+0000","last_undegraded":"2024-05-10T21:08:22.237859+0000","last_fullsized":"2024-05-10T21:08:22.237859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893043+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.893043+0000","last_peered":"2024-05-10T21:04:56.893043+0000","last_clean":"2024-05-10T21:04:56.893043+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.893043+0000","last_undegraded":"2024-05-10T21:04:56.893043+0000","last_fullsized":"2024-05-10T21:04:56.893043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576512+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:25.576512+0000","last_peered":"2024-05-10T21:06:25.576512+0000","last_clean":"2024-05-10T21:06:25.576512+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:25.576512+0000","last_undegraded":"2024-05-10T21:06:25.576512+0000","last_fullsized":"2024-05-10T21:06:25.576512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895673+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895673+0000","last_peered":"2024-05-10T21:04:56.895673+0000","last_clean":"2024-05-10T21:04:56.895673+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895673+0000","last_undegraded":"2024-05-10T21:04:56.895673+0000","last_fullsized":"2024-05-10T21:04:56.895673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578224+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:25.578224+0000","last_peered":"2024-05-10T21:06:25.578224+0000","last_clean":"2024-05-10T21:06:25.578224+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:25.578224+0000","last_undegraded":"2024-05-10T21:06:25.578224+0000","last_fullsized":"2024-05-10T21:06:25.578224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555147+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.555147+0000","last_peered":"2024-05-10T21:06:25.555147+0000","last_clean":"2024-05-10T21:06:25.555147+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.555147+0000","last_undegraded":"2024-05-10T21:06:25.555147+0000","last_fullsized":"2024-05-10T21:06:25.555147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:20.233124+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:20.233124+0000","last_peered":"2024-05-10T21:08:20.233124+0000","last_clean":"2024-05-10T21:08:20.233124+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:20.233124+0000","last_undegraded":"2024-05-10T21:08:20.233124+0000","last_fullsized":"2024-05-10T21:08:20.233124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570231+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:25.570231+0000","last_peered":"2024-05-10T21:06:25.570231+0000","last_clean":"2024-05-10T21:06:25.570231+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:25.570231+0000","last_undegraded":"2024-05-10T21:06:25.570231+0000","last_fullsized":"2024-05-10T21:06:25.570231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331885+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331885+0000","last_peered":"2024-05-10T21:04:57.331885+0000","last_clean":"2024-05-10T21:04:57.331885+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331885+0000","last_undegraded":"2024-05-10T21:04:57.331885+0000","last_fullsized":"2024-05-10T21:04:57.331885+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552134+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:25.552134+0000","last_peered":"2024-05-10T21:06:25.552134+0000","last_clean":"2024-05-10T21:06:25.552134+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:25.552134+0000","last_undegraded":"2024-05-10T21:06:25.552134+0000","last_fullsized":"2024-05-10T21:06:25.552134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891596+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891596+0000","last_peered":"2024-05-10T21:04:56.891596+0000","last_clean":"2024-05-10T21:04:56.891596+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891596+0000","last_undegraded":"2024-05-10T21:04:56.891596+0000","last_fullsized":"2024-05-10T21:04:56.891596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562727+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:25.562727+0000","last_peered":"2024-05-10T21:06:25.562727+0000","last_clean":"2024-05-10T21:06:25.562727+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:25.562727+0000","last_undegraded":"2024-05-10T21:06:25.562727+0000","last_fullsized":"2024-05-10T21:06:25.562727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.918836+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:28.918836+0000","last_peered":"2024-05-10T21:06:28.918836+0000","last_clean":"2024-05-10T21:06:28.918836+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:28.918836+0000","last_undegraded":"2024-05-10T21:06:28.918836+0000","last_fullsized":"2024-05-10T21:06:28.918836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557980+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:25.557980+0000","last_peered":"2024-05-10T21:06:25.557980+0000","last_clean":"2024-05-10T21:06:25.557980+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:25.557980+0000","last_undegraded":"2024-05-10T21:06:25.557980+0000","last_fullsized":"2024-05-10T21:06:25.557980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552796+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:25.552796+0000","last_peered":"2024-05-10T21:06:25.552796+0000","last_clean":"2024-05-10T21:06:25.552796+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:25.552796+0000","last_undegraded":"2024-05-10T21:06:25.552796+0000","last_fullsized":"2024-05-10T21:06:25.552796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569414+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:25.569414+0000","last_peered":"2024-05-10T21:06:25.569414+0000","last_clean":"2024-05-10T21:06:25.569414+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:25.569414+0000","last_undegraded":"2024-05-10T21:06:25.569414+0000","last_fullsized":"2024-05-10T21:06:25.569414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579761+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:25.579761+0000","last_peered":"2024-05-10T21:06:25.579761+0000","last_clean":"2024-05-10T21:06:25.579761+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:25.579761+0000","last_undegraded":"2024-05-10T21:06:25.579761+0000","last_fullsized":"2024-05-10T21:06:25.579761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74594,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1327,"num_read_kb":742,"num_write":181,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037836,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430298,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:25.277 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T21:08:25.277 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T21:08:25.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.605+0000 7f083d200640 1 -- 172.21.15.50:0/1701572753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 msgr2=0x7f083810d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.605+0000 7f083d200640 1 --2- 172.21.15.50:0/1701572753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f083810d0f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0828009900 tx=0x7f082802a3f0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 -- 172.21.15.50:0/1701572753 shutdown_connections 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 --2- 172.21.15.50:0/1701572753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083810d630 0x7f083810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 --2- 172.21.15.50:0/1701572753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0838112330 0x7f0838114750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 --2- 172.21.15.50:0/1701572753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f083810d0f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 -- 172.21.15.50:0/1701572753 >> 172.21.15.50:0/1701572753 conn(0x7f0838072810 msgr2=0x7f0838109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 -- 172.21.15.50:0/1701572753 shutdown_connections 2024-05-10T21:08:25.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.606+0000 7f083d200640 1 -- 172.21.15.50:0/1701572753 wait complete. 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.607+0000 7f083d200640 1 Processor -- start 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.607+0000 7f083d200640 1 -- start start 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f08381a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0838112330 0x7f08381ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0838117f60 con 0x7f083810d630 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0838117de0 con 0x7f083810c3f0 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f083d200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08381180e0 con 0x7f0838112330 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f0835400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:25.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f0835400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47444/0 (socket says 172.21.15.50:47444) 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f0835400640 1 -- 172.21.15.50:0/3678213158 learned_addr learned my addr 172.21.15.50:0/3678213158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f0835e00640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f08381a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.608+0000 7f0836800640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0838112330 0x7f08381ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 -- 172.21.15.50:0/3678213158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0838112330 msgr2=0x7f08381ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0838112330 0x7f08381ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 -- 172.21.15.50:0/3678213158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 msgr2=0x7f08381a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f08381a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 -- 172.21.15.50:0/3678213158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08381ac8d0 con 0x7f083810d630 2024-05-10T21:08:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f0835400640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9f0 tx=0x7f082400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082400ccc0 con 0x7f083810d630 2024-05-10T21:08:25.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.609+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381acbc0 con 0x7f083810d630 2024-05-10T21:08:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.610+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082400ce60 con 0x7f083810d630 2024-05-10T21:08:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.610+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824010430 con 0x7f083810d630 2024-05-10T21:08:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.610+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381ad110 con 0x7f083810d630 2024-05-10T21:08:25.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.611+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08381ad340 con 0x7f083810d630 2024-05-10T21:08:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.614+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f08240046c0 con 0x7f083810d630 2024-05-10T21:08:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.615+0000 7f082ea00640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 0x7f081c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.615+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f08240e2820 con 0x7f083810d630 2024-05-10T21:08:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.616+0000 7f0835e00640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 0x7f081c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.616+0000 7f0835e00640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 0x7f081c0b4d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f082802a9e0 tx=0x7f08280314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.617+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0824054df0 con 0x7f083810d630 2024-05-10T21:08:25.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:25 smithi142 ceph-mon[34846]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:25.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:25 smithi050 ceph-mon[29546]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.824+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08381ad510 con 0x7f081c0b27f0 2024-05-10T21:08:25.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.849+0000 7f082ea00640 1 -- 172.21.15.50:0/3678213158 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258823 (secure 0 0 0) 0x7f08381ad510 con 0x7f081c0b27f0 2024-05-10T21:08:25.851 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:25.854 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 msgr2=0x7f081c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 0x7f081c0b4d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f082802a9e0 tx=0x7f08280314a0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 msgr2=0x7f08381a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9f0 tx=0x7f082400edf0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 shutdown_connections 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f081c0b27f0 0x7f081c0b4d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0838112330 0x7f08381ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f083810d630 0x7f08381a52d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 --2- 172.21.15.50:0/3678213158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f083810c3f0 0x7f08381a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.855+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 >> 172.21.15.50:0/3678213158 conn(0x7f0838072810 msgr2=0x7f0838112740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.856+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 shutdown_connections 2024-05-10T21:08:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:25.856+0000 7f083d200640 1 -- 172.21.15.50:0/3678213158 wait complete. 2024-05-10T21:08:25.869 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:25 smithi092 ceph-mon[34415]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:25.885 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-05-10T21:08:25.676048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665196,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1403,"num_read_kb":806,"num_write":306,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5993,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006222"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578816+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:25.578816+0000","last_peered":"2024-05-10T21:06:25.578816+0000","last_clean":"2024-05-10T21:06:25.578816+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:25.578816+0000","last_undegraded":"2024-05-10T21:06:25.578816+0000","last_fullsized":"2024-05-10T21:06:25.578816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580291+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:25.580291+0000","last_peered":"2024-05-10T21:06:25.580291+0000","last_clean":"2024-05-10T21:06:25.580291+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:25.580291+0000","last_undegraded":"2024-05-10T21:06:25.580291+0000","last_fullsized":"2024-05-10T21:06:25.580291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581398+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:25.581398+0000","last_peered":"2024-05-10T21:06:25.581398+0000","last_clean":"2024-05-10T21:06:25.581398+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:25.581398+0000","last_undegraded":"2024-05-10T21:06:25.581398+0000","last_fullsized":"2024-05-10T21:06:25.581398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566835+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566835+0000","last_peered":"2024-05-10T21:06:25.566835+0000","last_clean":"2024-05-10T21:06:25.566835+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566835+0000","last_undegraded":"2024-05-10T21:06:25.566835+0000","last_fullsized":"2024-05-10T21:06:25.566835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.184551+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:04:57.184551+0000","last_peered":"2024-05-10T21:04:57.184551+0000","last_clean":"2024-05-10T21:04:57.184551+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:04:57.184551+0000","last_undegraded":"2024-05-10T21:04:57.184551+0000","last_fullsized":"2024-05-10T21:04:57.184551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572865+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.572865+0000","last_peered":"2024-05-10T21:06:25.572865+0000","last_clean":"2024-05-10T21:06:25.572865+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.572865+0000","last_undegraded":"2024-05-10T21:06:25.572865+0000","last_fullsized":"2024-05-10T21:06:25.572865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581364+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:25.581364+0000","last_peered":"2024-05-10T21:06:25.581364+0000","last_clean":"2024-05-10T21:06:25.581364+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:25.581364+0000","last_undegraded":"2024-05-10T21:06:25.581364+0000","last_fullsized":"2024-05-10T21:06:25.581364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":250,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:19.087787+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:19.087787+0000","last_peered":"2024-05-10T21:08:19.087787+0000","last_clean":"2024-05-10T21:08:19.087787+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:19.087787+0000","last_undegraded":"2024-05-10T21:08:19.087787+0000","last_fullsized":"2024-05-10T21:08:19.087787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:21.090583+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:21.090583+0000","last_peered":"2024-05-10T21:08:21.090583+0000","last_clean":"2024-05-10T21:08:21.090583+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:21.090583+0000","last_undegraded":"2024-05-10T21:08:21.090583+0000","last_fullsized":"2024-05-10T21:08:21.090583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":510,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.092334+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:22.092334+0000","last_peered":"2024-05-10T21:08:22.092334+0000","last_clean":"2024-05-10T21:08:22.092334+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:22.092334+0000","last_undegraded":"2024-05-10T21:08:22.092334+0000","last_fullsized":"2024-05-10T21:08:22.092334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70142,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":446,"num_read_kb":292,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":253,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:22.237859+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:22.237859+0000","last_peered":"2024-05-10T21:08:22.237859+0000","last_clean":"2024-05-10T21:08:22.237859+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:22.237859+0000","last_undegraded":"2024-05-10T21:08:22.237859+0000","last_fullsized":"2024-05-10T21:08:22.237859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.893043+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:04:56.893043+0000","last_peered":"2024-05-10T21:04:56.893043+0000","last_clean":"2024-05-10T21:04:56.893043+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:04:56.893043+0000","last_undegraded":"2024-05-10T21:04:56.893043+0000","last_fullsized":"2024-05-10T21:04:56.893043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576512+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:25.576512+0000","last_peered":"2024-05-10T21:06:25.576512+0000","last_clean":"2024-05-10T21:06:25.576512+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:25.576512+0000","last_undegraded":"2024-05-10T21:06:25.576512+0000","last_fullsized":"2024-05-10T21:06:25.576512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895673+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:04:56.895673+0000","last_peered":"2024-05-10T21:04:56.895673+0000","last_clean":"2024-05-10T21:04:56.895673+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:04:56.895673+0000","last_undegraded":"2024-05-10T21:04:56.895673+0000","last_fullsized":"2024-05-10T21:04:56.895673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578224+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:25.578224+0000","last_peered":"2024-05-10T21:06:25.578224+0000","last_clean":"2024-05-10T21:06:25.578224+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:25.578224+0000","last_undegraded":"2024-05-10T21:06:25.578224+0000","last_fullsized":"2024-05-10T21:06:25.578224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.555147+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.555147+0000","last_peered":"2024-05-10T21:06:25.555147+0000","last_clean":"2024-05-10T21:06:25.555147+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.555147+0000","last_undegraded":"2024-05-10T21:06:25.555147+0000","last_fullsized":"2024-05-10T21:06:25.555147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:20.233124+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:20.233124+0000","last_peered":"2024-05-10T21:08:20.233124+0000","last_clean":"2024-05-10T21:08:20.233124+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:20.233124+0000","last_undegraded":"2024-05-10T21:08:20.233124+0000","last_fullsized":"2024-05-10T21:08:20.233124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570231+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:25.570231+0000","last_peered":"2024-05-10T21:06:25.570231+0000","last_clean":"2024-05-10T21:06:25.570231+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:25.570231+0000","last_undegraded":"2024-05-10T21:06:25.570231+0000","last_fullsized":"2024-05-10T21:06:25.570231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331885+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:04:57.331885+0000","last_peered":"2024-05-10T21:04:57.331885+0000","last_clean":"2024-05-10T21:04:57.331885+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:04:57.331885+0000","last_undegraded":"2024-05-10T21:04:57.331885+0000","last_fullsized":"2024-05-10T21:04:57.331885+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552134+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:25.552134+0000","last_peered":"2024-05-10T21:06:25.552134+0000","last_clean":"2024-05-10T21:06:25.552134+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:25.552134+0000","last_undegraded":"2024-05-10T21:06:25.552134+0000","last_fullsized":"2024-05-10T21:06:25.552134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891596+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:04:56.891596+0000","last_peered":"2024-05-10T21:04:56.891596+0000","last_clean":"2024-05-10T21:04:56.891596+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:04:56.891596+0000","last_undegraded":"2024-05-10T21:04:56.891596+0000","last_fullsized":"2024-05-10T21:04:56.891596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562727+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:25.562727+0000","last_peered":"2024-05-10T21:06:25.562727+0000","last_clean":"2024-05-10T21:06:25.562727+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:25.562727+0000","last_undegraded":"2024-05-10T21:06:25.562727+0000","last_fullsized":"2024-05-10T21:06:25.562727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.918836+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:28.918836+0000","last_peered":"2024-05-10T21:06:28.918836+0000","last_clean":"2024-05-10T21:06:28.918836+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:28.918836+0000","last_undegraded":"2024-05-10T21:06:28.918836+0000","last_fullsized":"2024-05-10T21:06:28.918836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557980+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:25.557980+0000","last_peered":"2024-05-10T21:06:25.557980+0000","last_clean":"2024-05-10T21:06:25.557980+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:25.557980+0000","last_undegraded":"2024-05-10T21:06:25.557980+0000","last_fullsized":"2024-05-10T21:06:25.557980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.552796+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:25.552796+0000","last_peered":"2024-05-10T21:06:25.552796+0000","last_clean":"2024-05-10T21:06:25.552796+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:25.552796+0000","last_undegraded":"2024-05-10T21:06:25.552796+0000","last_fullsized":"2024-05-10T21:06:25.552796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569414+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:25.569414+0000","last_peered":"2024-05-10T21:06:25.569414+0000","last_clean":"2024-05-10T21:06:25.569414+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:25.569414+0000","last_undegraded":"2024-05-10T21:06:25.569414+0000","last_fullsized":"2024-05-10T21:06:25.569414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579761+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:25.579761+0000","last_peered":"2024-05-10T21:06:25.579761+0000","last_clean":"2024-05-10T21:06:25.579761+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:25.579761+0000","last_undegraded":"2024-05-10T21:06:25.579761+0000","last_fullsized":"2024-05-10T21:06:25.579761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74594,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1327,"num_read_kb":742,"num_write":181,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037837,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234068,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430299,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822756,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:25.890 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:08:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:26.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:26 smithi050 ceph-mon[29546]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:26.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:26 smithi050 ceph-mon[29546]: pgmap v447: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-10T21:08:26.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:26 smithi050 ceph-mon[29546]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:26.869 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:26 smithi092 ceph-mon[34415]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:26.869 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:26 smithi092 ceph-mon[34415]: pgmap v447: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-10T21:08:26.869 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:26 smithi092 ceph-mon[34415]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:26.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:26 smithi142 ceph-mon[34846]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:26.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:26 smithi142 ceph-mon[34846]: pgmap v447: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-10T21:08:26.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:26 smithi142 ceph-mon[34846]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:28.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 -- 172.21.15.50:0/380981180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c109870 msgr2=0x7fba1c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:28.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 --2- 172.21.15.50:0/380981180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c109870 0x7fba1c1111d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b0d0 tx=0x7fba1400b3e0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.758 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:28 smithi050 ceph-mon[29546]: pgmap v448: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 -- 172.21.15.50:0/380981180 shutdown_connections 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 --2- 172.21.15.50:0/380981180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c109870 0x7fba1c1111d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 --2- 172.21.15.50:0/380981180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 --2- 172.21.15.50:0/380981180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c108f20 0x7fba1c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.756+0000 7fba24400700 1 -- 172.21.15.50:0/380981180 >> 172.21.15.50:0/380981180 conn(0x7fba1c100040 msgr2=0x7fba1c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.757+0000 7fba24400700 1 -- 172.21.15.50:0/380981180 shutdown_connections 2024-05-10T21:08:28.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.757+0000 7fba24400700 1 -- 172.21.15.50:0/380981180 wait complete. 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.758+0000 7fba24400700 1 Processor -- start 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.758+0000 7fba24400700 1 -- start start 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 0x7fba1c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c109870 0x7fba1c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1144f0 con 0x7fba1c108f20 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114370 con 0x7fba1c109870 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba24400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114670 con 0x7fba1c1045f0 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba21a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba21a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56952/0 (socket says 172.21.15.50:56952) 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba21a00700 1 -- 172.21.15.50:0/1568762619 learned_addr learned my addr 172.21.15.50:0/1568762619 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba21000700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 0x7fba1c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:28.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.759+0000 7fba22400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c109870 0x7fba1c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:28.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 -- 172.21.15.50:0/1568762619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c109870 msgr2=0x7fba1c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:28.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c109870 0x7fba1c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 -- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 msgr2=0x7fba1c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 0x7fba1c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 -- 172.21.15.50:0/1568762619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c1a9200 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21000700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 0x7fba1c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba21a00700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1000e9a0 tx=0x7fba1000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1000ca10 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1a94f0 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.760+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba1000cbb0 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.761+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a9a80 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.761+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10018620 con 0x7fba1c1045f0 2024-05-10T21:08:28.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.762+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c0512c0 con 0x7fba1c1045f0 2024-05-10T21:08:28.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.764+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba1004aab0 con 0x7fba1c1045f0 2024-05-10T21:08:28.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.766+0000 7fba1a000700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 0x7fba040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.766+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fba100e2450 con 0x7fba1c1045f0 2024-05-10T21:08:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.766+0000 7fba21000700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 0x7fba040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:28.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.767+0000 7fba21000700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 0x7fba040b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba1c1077f0 tx=0x7fba08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:28.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:28.768+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba100aa140 con 0x7fba1c1045f0 2024-05-10T21:08:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.062+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba1c076ff0 con 0x7fba1c1045f0 2024-05-10T21:08:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.063+0000 7fba1a000700 1 -- 172.21.15.50:0/1568762619 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fba100a9a40 con 0x7fba1c1045f0 2024-05-10T21:08:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:29.064 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:08:29.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 msgr2=0x7fba040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:29.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 0x7fba040b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba1c1077f0 tx=0x7fba08009260 comp rx=0 tx=0).stop 2024-05-10T21:08:29.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 msgr2=0x7fba1c075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:29.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1000e9a0 tx=0x7fba1000ecb0 comp rx=0 tx=0).stop 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 shutdown_connections 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fba040b2880 0x7fba040b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba1c109870 0x7fba1c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba1c1045f0 0x7fba1c075ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 --2- 172.21.15.50:0/1568762619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba1c108f20 0x7fba1c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 >> 172.21.15.50:0/1568762619 conn(0x7fba1c100040 msgr2=0x7fba1c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 shutdown_connections 2024-05-10T21:08:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:29.067+0000 7fba24400700 1 -- 172.21.15.50:0/1568762619 wait complete. 2024-05-10T21:08:29.069 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:08:29.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:28 smithi092 ceph-mon[34415]: pgmap v448: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:28 smithi142 ceph-mon[34846]: pgmap v448: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:08:29.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:08:30.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:29 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1568762619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:30.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:29 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1568762619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:30.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:29 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1568762619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:30.429 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:30.894 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:30 smithi050 ceph-mon[29546]: pgmap v449: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T21:08:31.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:30 smithi092 ceph-mon[34415]: pgmap v449: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T21:08:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:30 smithi142 ceph-mon[34846]: pgmap v449: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 -- 172.21.15.50:0/3190170341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 msgr2=0x7fadc0113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 --2- 172.21.15.50:0/3190170341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc0113000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fadb800b0a0 tx=0x7fadb800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 -- 172.21.15.50:0/3190170341 shutdown_connections 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 --2- 172.21.15.50:0/3190170341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc0113000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 --2- 172.21.15.50:0/3190170341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc0076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 --2- 172.21.15.50:0/3190170341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 -- 172.21.15.50:0/3190170341 >> 172.21.15.50:0/3190170341 conn(0x7fadc0100040 msgr2=0x7fadc0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 -- 172.21.15.50:0/3190170341 shutdown_connections 2024-05-10T21:08:32.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.704+0000 7fadc5600700 1 -- 172.21.15.50:0/3190170341 wait complete. 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.705+0000 7fadc5600700 1 Processor -- start 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.705+0000 7fadc5600700 1 -- start start 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc01166d0 con 0x7fadc0075d30 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0116550 con 0x7fadc0076680 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadc5600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0116850 con 0x7fadc0110ba0 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadbea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadbea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47488/0 (socket says 172.21.15.50:47488) 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadbea00700 1 -- 172.21.15.50:0/3988163562 learned_addr learned my addr 172.21.15.50:0/3988163562 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadbe000700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.706+0000 7fadbf400700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 -- 172.21.15.50:0/3988163562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 msgr2=0x7fadc01a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc01a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 -- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 msgr2=0x7fadc019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 -- 172.21.15.50:0/3988163562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01a1ef0 con 0x7fadc0076680 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbea00700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:32.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.707+0000 7fadbe000700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc019d5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9f0 tx=0x7fadac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:32.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.708+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00c9c0 con 0x7fadc0076680 2024-05-10T21:08:32.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.708+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01a21e0 con 0x7fadc0076680 2024-05-10T21:08:32.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.708+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac00cb60 con 0x7fadc0076680 2024-05-10T21:08:32.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.708+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac018680 con 0x7fadc0076680 2024-05-10T21:08:32.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.708+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01a9b00 con 0x7fadc0076680 2024-05-10T21:08:32.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.709+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc00512c0 con 0x7fadc0076680 2024-05-10T21:08:32.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.712+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadac04aab0 con 0x7fadc0076680 2024-05-10T21:08:32.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.713+0000 7fadb7400700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 0x7fada40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:32.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.713+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fadac0e24a0 con 0x7fadc0076680 2024-05-10T21:08:32.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.714+0000 7fadbea00700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 0x7fada40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:32.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.714+0000 7fadbea00700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 0x7fada40b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadb0011550 tx=0x7fadb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:32.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.716+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadac0aa190 con 0x7fadc0076680 2024-05-10T21:08:32.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.959+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadc002e5f0 con 0x7fadc0076680 2024-05-10T21:08:32.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:32 smithi050 ceph-mon[29546]: pgmap v450: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:08:32.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.961+0000 7fadb7400700 1 -- 172.21.15.50:0/3988163562 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7fadac0a5070 con 0x7fadc0076680 2024-05-10T21:08:32.962 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:32.963 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:08:32.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 msgr2=0x7fada40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:32.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 0x7fada40b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadb0011550 tx=0x7fadb0009260 comp rx=0 tx=0).stop 2024-05-10T21:08:32.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 msgr2=0x7fadc019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:32.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc019d5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9f0 tx=0x7fadac00ed00 comp rx=0 tx=0).stop 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 shutdown_connections 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fada40b2880 0x7fada40b4d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc0110ba0 0x7fadc01a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc0075d30 0x7fadc019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 --2- 172.21.15.50:0/3988163562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadc0076680 0x7fadc019d5c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 >> 172.21.15.50:0/3988163562 conn(0x7fadc0100040 msgr2=0x7fadc0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 shutdown_connections 2024-05-10T21:08:32.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:32.966+0000 7fadc5600700 1 -- 172.21.15.50:0/3988163562 wait complete. 2024-05-10T21:08:33.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:32 smithi092 ceph-mon[34415]: pgmap v450: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:08:33.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:32 smithi142 ceph-mon[34846]: pgmap v450: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T21:08:33.866 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T21:08:33.866 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-10T21:08:33.890 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:08:33.890 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-10T21:08:33.891 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T21:08:33.891 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:08:33.891 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:08:33.899 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-10T21:08:33.911 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T21:08:33.911 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:08:34.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:33 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3988163562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:34.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:33 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3988163562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:34.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:33 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3988163562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:08:34.352 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:34.362 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:35.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:34 smithi050 ceph-mon[29546]: pgmap v451: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T21:08:35.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:34 smithi092 ceph-mon[34415]: pgmap v451: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T21:08:35.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:34 smithi142 ceph-mon[34846]: pgmap v451: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T21:08:37.059 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:36 smithi050 ceph-mon[29546]: pgmap v452: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:37.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:36 smithi092 ceph-mon[34415]: pgmap v452: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:37.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:36 smithi142 ceph-mon[34846]: pgmap v452: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:08:37.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 -- 172.21.15.50:0/832947078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 msgr2=0x7ff13810e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 --2- 172.21.15.50:0/832947078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff13810e150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff120009990 tx=0x7ff120009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 -- 172.21.15.50:0/832947078 shutdown_connections 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 --2- 172.21.15.50:0/832947078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff13810e720 0x7ff138110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 --2- 172.21.15.50:0/832947078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1381052b0 0x7ff1381056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 --2- 172.21.15.50:0/832947078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff13810e150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 -- 172.21.15.50:0/832947078 >> 172.21.15.50:0/832947078 conn(0x7ff138078c40 msgr2=0x7ff138079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 -- 172.21.15.50:0/832947078 shutdown_connections 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.058+0000 7ff13d400700 1 -- 172.21.15.50:0/832947078 wait complete. 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 Processor -- start 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 -- start start 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1381052b0 0x7ff138102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 0x7ff138103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381142d0 con 0x7ff138105c00 2024-05-10T21:08:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114150 con 0x7ff1381052b0 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.060+0000 7ff13d400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114450 con 0x7ff13810e720 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59088/0 (socket says 172.21.15.50:59088) 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff13ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 0x7ff138103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 -- 172.21.15.50:0/3416784274 learned_addr learned my addr 172.21.15.50:0/3416784274 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff137e00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1381052b0 0x7ff138102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 -- 172.21.15.50:0/3416784274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 msgr2=0x7ff138103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 0x7ff138103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 -- 172.21.15.50:0/3416784274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1381052b0 msgr2=0x7ff138102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1381052b0 0x7ff138102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff12fe00700 1 -- 172.21.15.50:0/3416784274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a9060 con 0x7ff138105c00 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.061+0000 7ff13ca00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 0x7ff138103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff12fe00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff120005910 tx=0x7ff12000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff135600700 1 -- 172.21.15.50:0/3416784274 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120027070 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff135600700 1 -- 172.21.15.50:0/3416784274 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff120022470 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a92f0 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff135600700 1 -- 172.21.15.50:0/3416784274 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120017630 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.062+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a97a0 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.066+0000 7ff135600700 1 -- 172.21.15.50:0/3416784274 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff12002a070 con 0x7ff138105c00 2024-05-10T21:08:37.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.067+0000 7ff14d5e6b80 1 Processor -- start 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.067+0000 7ff14d5e6b80 1 -- start start 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.067+0000 7ff135600700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff135600700 1 -- 172.21.15.50:0/3416784274 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff1200e2e60 con 0x7ff138105c00 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff13ca00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e8a01eaaa0 con 0x55e8a014aec0 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e8a01887e0 con 0x55e8a02ceb00 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8a01e6a60 con 0x55e8a02c8cc0 2024-05-10T21:08:37.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff137e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff13ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54028/0 (socket says 172.21.15.50:54028) 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.068+0000 7ff12fe00700 1 -- 172.21.15.50:0/1406133495 learned_addr learned my addr 172.21.15.50:0/1406133495 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 -- 172.21.15.50:0/1406133495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 msgr2=0x55e8a02c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 -- 172.21.15.50:0/1406133495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 msgr2=0x55e8a02ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 -- 172.21.15.50:0/1406133495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a0137950 con 0x55e8a02ceb00 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff13ca00700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 0x7ff11c0b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff130009fd0 tx=0x7ff130007b20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.069+0000 7ff12fe00700 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1200aa470 tx=0x7ff120009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4bff551ba257fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.070+0000 7ff12d600700 1 -- 172.21.15.50:0/1406133495 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200aa4e0 con 0x55e8a02ceb00 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.070+0000 7ff12d600700 1 -- 172.21.15.50:0/1406133495 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1200ae890 con 0x55e8a02ceb00 2024-05-10T21:08:37.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.070+0000 7ff12d600700 1 -- 172.21.15.50:0/1406133495 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200aa9a0 con 0x55e8a02ceb00 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/1406133495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 msgr2=0x55e8a02c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1200aa470 tx=0x7ff120009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/1406133495 shutdown_connections 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/1406133495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02c8780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/1406133495 >> 172.21.15.50:0/1406133495 conn(0x55e8a018fff0 msgr2=0x55e8a02c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/1406133495 shutdown_connections 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/1406133495 wait complete. 2024-05-10T21:08:37.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.071+0000 7ff14d5e6b80 1 -- start start 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 Processor -- start 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e8a02d52d0 con 0x55e8a02c8cc0 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e8a0150ba0 con 0x55e8a02ceb00 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8a02d5450 con 0x55e8a014aec0 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.072+0000 7ff14d5e6b80 10 client.0 _ll_register_callbacks cb 0x55e8a0191cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45332/0 (socket says 172.21.15.50:45332) 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 -- 172.21.15.50:0/2062075300 learned_addr learned my addr 172.21.15.50:0/2062075300 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:37.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff13ca00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 -- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 msgr2=0x55e8a02d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 -- 172.21.15.50:0/2062075300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a02c8cc0 msgr2=0x55e8a02d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.073+0000 7ff137e00700 1 -- 172.21.15.50:0/2062075300 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a01b04a0 con 0x55e8a014aec0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff137e00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1280127f0 tx=0x7ff128012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb31605740962188 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff126000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12800f4e0 con 0x55e8a014aec0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1200ae890 con 0x55e8a014aec0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff12800f680 con 0x55e8a014aec0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff12800f880 con 0x55e8a014aec0 2024-05-10T21:08:37.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff14d5e6b80 10 client.25117 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.074+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8a02dfe50 con 0x55e8a014aec0 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1200ae890 con 0x55e8a014aec0 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff128010960 con 0x55e8a014aec0 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 10 client.25117 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 10 client.25117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 20 client.25117 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02e2490 0x55e8a02e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 4 client.25117 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a0295240 con 0x55e8a02e2490 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.075+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02e2490 0x55e8a02e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.076+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02e2490 0x55e8a02e4900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc28b497d8315dd8 server_cookie=701f3c5364e7e5e4 in_seq=0 out_seq=0 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.076+0000 7ff126000700 10 client.25117 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff126000700 1 -- 172.21.15.50:0/2062075300 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8a0295240 con 0x55e8a02e2490 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff126000700 10 client.25117 handle_command_reply: tid=0 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 client.25117 shutdown 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 msgr2=0x55e8a02d2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff1280127f0 tx=0x7ff128012b00 comp rx=0 tx=0).stop 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 shutdown_connections 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02e2490 0x55e8a02e4900 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02ceb00 0x55e8a02d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a014aec0 0x55e8a02d2250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff126000700 0 client.25117 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2062075300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 >> 172.21.15.50:0/2062075300 conn(0x55e8a018fff0 msgr2=0x55e8a02c7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 shutdown_connections 2024-05-10T21:08:37.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.077+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2062075300 wait complete. 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.078+0000 7ff14d5e6b80 20 client.25117 trim_cache size 0 max 16384 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 Processor -- start 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 -- start start 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 0x55e8a02c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381b3340 con 0x55e8a014aec0 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381b0bd0 con 0x55e8a02c8cc0 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.104+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381aeaa0 con 0x55e8a018fff0 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 0x55e8a02c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02bdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59122/0 (socket says 172.21.15.50:59122) 2024-05-10T21:08:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff12fe00700 1 -- 172.21.15.50:0/4012455720 learned_addr learned my addr 172.21.15.50:0/4012455720 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff13ca00700 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 -- 172.21.15.50:0/4012455720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02c8cc0 msgr2=0x55e8a02d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 -- 172.21.15.50:0/4012455720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 msgr2=0x55e8a02bdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02bdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.105+0000 7ff137e00700 1 -- 172.21.15.50:0/4012455720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a02c9460 con 0x55e8a018fff0 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.106+0000 7ff137e00700 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 0x55e8a02c8780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1280127f0 tx=0x7ff128005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc961b11f37e73e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.106+0000 7ff12cc00700 1 -- 172.21.15.50:0/4012455720 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12800f620 con 0x55e8a018fff0 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.106+0000 7ff12cc00700 1 -- 172.21.15.50:0/4012455720 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1280050b0 con 0x55e8a018fff0 2024-05-10T21:08:37.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.106+0000 7ff12cc00700 1 -- 172.21.15.50:0/4012455720 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128010690 con 0x55e8a018fff0 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/4012455720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 msgr2=0x55e8a02c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 0x55e8a02c8780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1280127f0 tx=0x7ff128005a40 comp rx=0 tx=0).stop 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/4012455720 shutdown_connections 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a018fff0 0x55e8a02c8780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/4012455720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/4012455720 >> 172.21.15.50:0/4012455720 conn(0x55e8a02cdc80 msgr2=0x55e8a02d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/4012455720 shutdown_connections 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.107+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/4012455720 wait complete. 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 -- start start 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 Processor -- start 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381af0f0 con 0x55e8a014aec0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381b0500 con 0x55e8a018fff0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381af760 con 0x55e8a02c8cc0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.108+0000 7ff14d5e6b80 10 client.0 _ll_register_callbacks cb 0x55e8a0191cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff137e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54058/0 (socket says 172.21.15.50:54058) 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 -- 172.21.15.50:0/2558829485 learned_addr learned my addr 172.21.15.50:0/2558829485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff137e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02dab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59130/0 (socket says 172.21.15.50:59130) 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff13ca00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 -- 172.21.15.50:0/2558829485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 msgr2=0x55e8a02d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 -- 172.21.15.50:0/2558829485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 msgr2=0x55e8a02dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:37.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 -- 172.21.15.50:0/2558829485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a02d8290 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff12fe00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1200a9dc0 tx=0x7ff12000b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9be260a9f728 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.109+0000 7ff126000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200ae890 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1280050b0 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff120022940 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1200aa740 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff14d5e6b80 10 client.25130 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8a034db90 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1280050b0 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.110+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff1200b2e20 con 0x55e8a018fff0 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 10 client.25130 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 10 client.25130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:37.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 20 client.25130 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02ceb00 0x55e8a03501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 4 client.25130 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8a01f8480 con 0x55e8a02ceb00 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff137e00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02ceb00 0x55e8a03501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff137e00700 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02ceb00 0x55e8a03501a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02dc3d4f873ae8a server_cookie=a4c6f1b6ef353a1c in_seq=0 out_seq=0 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.111+0000 7ff126000700 10 client.25130 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff126000700 1 -- 172.21.15.50:0/2558829485 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8a01f8480 con 0x55e8a02ceb00 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff126000700 10 client.25130 handle_command_reply: tid=0 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 client.25130 shutdown 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 msgr2=0x55e8a02d7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1200a9dc0 tx=0x7ff12000b310 comp rx=0 tx=0).stop 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 shutdown_connections 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e8a02ceb00 0x55e8a03501a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8a02c8cc0 0x55e8a02d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8a014aec0 0x55e8a02dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 --2- 172.21.15.50:0/2558829485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e8a018fff0 0x55e8a02d7670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff126000700 0 client.25130 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.113+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 >> 172.21.15.50:0/2558829485 conn(0x55e8a02cdc80 msgr2=0x55e8a02d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "ad893aa3-68dc-45e0-a690-d6b353a211d5", 2024-05-10T21:08:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 shutdown_connections 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff14d5e6b80 1 -- 172.21.15.50:0/2558829485 wait complete. 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff14d5e6b80 20 client.25130 trim_cache size 0 max 16384 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 msgr2=0x7ff11c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 0x7ff11c0b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff130009fd0 tx=0x7ff130007b20 comp rx=0 tx=0).stop 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 msgr2=0x7ff1381034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.114+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff120005910 tx=0x7ff12000b670 comp rx=0 tx=0).stop 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 shutdown_connections 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff11c0b2900 0x7ff11c0b4e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1381052b0 0x7ff138102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13810e720 0x7ff138103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 --2- 172.21.15.50:0/3416784274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff138105c00 0x7ff1381034e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 >> 172.21.15.50:0/3416784274 conn(0x7ff138078c40 msgr2=0x7ff1380ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 shutdown_connections 2024-05-10T21:08:37.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:37.115+0000 7ff13d400700 1 -- 172.21.15.50:0/3416784274 wait complete. 2024-05-10T21:08:38.118 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:37 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:08:38.118 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.118 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:08:38.118 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:37 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:08:38.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:08:38.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:37 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:08:38.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:08:38.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:08:38.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:08:38.807 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 -- 172.21.15.50:0/457364225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 msgr2=0x7fb28c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 --2- 172.21.15.50:0/457364225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1100a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb284009fc0 tx=0x7fb28400b330 comp rx=0 tx=0).stop 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 -- 172.21.15.50:0/457364225 shutdown_connections 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 --2- 172.21.15.50:0/457364225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1100a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 --2- 172.21.15.50:0/457364225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c1034c0 0x7fb28c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 --2- 172.21.15.50:0/457364225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c107df0 0x7fb28c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:38.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 -- 172.21.15.50:0/457364225 >> 172.21.15.50:0/457364225 conn(0x7fb28c0fedc0 msgr2=0x7fb28c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:38.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 -- 172.21.15.50:0/457364225 shutdown_connections 2024-05-10T21:08:38.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.810+0000 7fb290e00700 1 -- 172.21.15.50:0/457364225 wait complete. 2024-05-10T21:08:38.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.812+0000 7fb290e00700 1 Processor -- start 2024-05-10T21:08:38.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.812+0000 7fb290e00700 1 -- start start 2024-05-10T21:08:38.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.812+0000 7fb290e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:38.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.812+0000 7fb290e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c107df0 0x7fb28c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:38.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb290e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:38.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb290e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c1133c0 con 0x7fb28c108740 2024-05-10T21:08:38.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb290e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c113240 con 0x7fb28c1034c0 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb290e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c113540 con 0x7fb28c107df0 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb289e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb289e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54082/0 (socket says 172.21.15.50:54082) 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb289e00700 1 -- 172.21.15.50:0/28201112 learned_addr learned my addr 172.21.15.50:0/28201112 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb289400700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c107df0 0x7fb28c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.813+0000 7fb28a800700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 -- 172.21.15.50:0/28201112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c107df0 msgr2=0x7fb28c19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c107df0 0x7fb28c19d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 -- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 msgr2=0x7fb28c1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 -- 172.21.15.50:0/28201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c1a1f20 con 0x7fb28c1034c0 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb289e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9c0 tx=0x7fb27c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c004a30 con 0x7fb28c1034c0 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27c00cc70 con 0x7fb28c1034c0 2024-05-10T21:08:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.814+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c1a2210 con 0x7fb28c1034c0 2024-05-10T21:08:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.815+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c005490 con 0x7fb28c1034c0 2024-05-10T21:08:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.815+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c1a9b30 con 0x7fb28c1034c0 2024-05-10T21:08:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.816+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c0512c0 con 0x7fb28c1034c0 2024-05-10T21:08:38.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.818+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb27c00c7c0 con 0x7fb28c1034c0 2024-05-10T21:08:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.820+0000 7fb282a00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 0x7fb2700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.820+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb27c0e2340 con 0x7fb28c1034c0 2024-05-10T21:08:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.820+0000 7fb289400700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 0x7fb2700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.821+0000 7fb289400700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 0x7fb2700b4d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb28c109890 tx=0x7fb278009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:38.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:38.824+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb27c0aa030 con 0x7fb28c1034c0 2024-05-10T21:08:39.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.068+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb28c10f470 con 0x7fb2700b2880 2024-05-10T21:08:39.069 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:38 smithi050 ceph-mon[29546]: pgmap v453: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-05-10T21:08:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.098+0000 7fb282a00700 1 -- 172.21.15.50:0/28201112 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258819 (secure 0 0 0) 0x7fb28c10f470 con 0x7fb2700b2880 2024-05-10T21:08:39.104 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 msgr2=0x7fb2700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 0x7fb2700b4d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb28c109890 tx=0x7fb278009260 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 msgr2=0x7fb28c19d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9c0 tx=0x7fb27c00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 shutdown_connections 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2700b2880 0x7fb2700b4d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb28c1034c0 0x7fb28c19d050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb28c108740 0x7fb28c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 --2- 172.21.15.50:0/28201112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb28c107df0 0x7fb28c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 >> 172.21.15.50:0/28201112 conn(0x7fb28c0fedc0 msgr2=0x7fb28c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 shutdown_connections 2024-05-10T21:08:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:39.104+0000 7fb290e00700 1 -- 172.21.15.50:0/28201112 wait complete. 2024-05-10T21:08:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:39.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:38 smithi092 ceph-mon[34415]: pgmap v453: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-05-10T21:08:39.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:38 smithi142 ceph-mon[34846]: pgmap v453: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-05-10T21:08:41.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:40 smithi050 ceph-mon[29546]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:41.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:40 smithi050 ceph-mon[29546]: pgmap v454: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:41.057 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-05-10T21:08:37.682203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665196,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1489,"num_read_kb":852,"num_write":306,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":46,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006154"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878346+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:31.878346+0000","last_peered":"2024-05-10T21:06:31.878346+0000","last_clean":"2024-05-10T21:06:31.878346+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:31.878346+0000","last_undegraded":"2024-05-10T21:06:31.878346+0000","last_fullsized":"2024-05-10T21:06:31.878346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813313+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:28.813313+0000","last_peered":"2024-05-10T21:06:28.813313+0000","last_clean":"2024-05-10T21:06:28.813313+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:28.813313+0000","last_undegraded":"2024-05-10T21:06:28.813313+0000","last_fullsized":"2024-05-10T21:06:28.813313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813290+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:28.813290+0000","last_peered":"2024-05-10T21:06:28.813290+0000","last_clean":"2024-05-10T21:06:28.813290+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:28.813290+0000","last_undegraded":"2024-05-10T21:06:28.813290+0000","last_fullsized":"2024-05-10T21:06:28.813290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566835+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566835+0000","last_peered":"2024-05-10T21:06:25.566835+0000","last_clean":"2024-05-10T21:06:25.566835+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566835+0000","last_undegraded":"2024-05-10T21:06:25.566835+0000","last_fullsized":"2024-05-10T21:06:25.566835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:23.917948+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:06:23.917948+0000","last_peered":"2024-05-10T21:06:23.917948+0000","last_clean":"2024-05-10T21:06:23.917948+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:06:23.917948+0000","last_undegraded":"2024-05-10T21:06:23.917948+0000","last_fullsized":"2024-05-10T21:06:23.917948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572865+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.572865+0000","last_peered":"2024-05-10T21:06:25.572865+0000","last_clean":"2024-05-10T21:06:25.572865+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.572865+0000","last_undegraded":"2024-05-10T21:06:25.572865+0000","last_fullsized":"2024-05-10T21:06:25.572865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813423+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:28.813423+0000","last_peered":"2024-05-10T21:06:28.813423+0000","last_clean":"2024-05-10T21:06:28.813423+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:28.813423+0000","last_undegraded":"2024-05-10T21:06:28.813423+0000","last_fullsized":"2024-05-10T21:06:28.813423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":265,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:34.111545+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:34.111545+0000","last_peered":"2024-05-10T21:08:34.111545+0000","last_clean":"2024-05-10T21:08:34.111545+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:34.111545+0000","last_undegraded":"2024-05-10T21:08:34.111545+0000","last_fullsized":"2024-05-10T21:08:34.111545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":232,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":262,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:36.113788+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:36.113788+0000","last_peered":"2024-05-10T21:08:36.113788+0000","last_clean":"2024-05-10T21:08:36.113788+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:36.113788+0000","last_undegraded":"2024-05-10T21:08:36.113788+0000","last_fullsized":"2024-05-10T21:08:36.113788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":231,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":541,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:37.115211+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:37.115211+0000","last_peered":"2024-05-10T21:08:37.115211+0000","last_clean":"2024-05-10T21:08:37.115211+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:37.115211+0000","last_undegraded":"2024-05-10T21:08:37.115211+0000","last_fullsized":"2024-05-10T21:08:37.115211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70142,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":308,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":263,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:32.260153+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:32.260153+0000","last_peered":"2024-05-10T21:08:32.260153+0000","last_clean":"2024-05-10T21:08:32.260153+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:32.260153+0000","last_undegraded":"2024-05-10T21:08:32.260153+0000","last_fullsized":"2024-05-10T21:08:32.260153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":232,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:23.917882+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:06:23.917882+0000","last_peered":"2024-05-10T21:06:23.917882+0000","last_clean":"2024-05-10T21:06:23.917882+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:06:23.917882+0000","last_undegraded":"2024-05-10T21:06:23.917882+0000","last_fullsized":"2024-05-10T21:06:23.917882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.440092+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:32.440092+0000","last_peered":"2024-05-10T21:06:32.440092+0000","last_clean":"2024-05-10T21:06:32.440092+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:32.440092+0000","last_undegraded":"2024-05-10T21:06:32.440092+0000","last_fullsized":"2024-05-10T21:06:32.440092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:24.928701+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:06:24.928701+0000","last_peered":"2024-05-10T21:06:24.928701+0000","last_clean":"2024-05-10T21:06:24.928701+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:06:24.928701+0000","last_undegraded":"2024-05-10T21:06:24.928701+0000","last_fullsized":"2024-05-10T21:06:24.928701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.440071+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:32.440071+0000","last_peered":"2024-05-10T21:06:32.440071+0000","last_clean":"2024-05-10T21:06:32.440071+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:32.440071+0000","last_undegraded":"2024-05-10T21:06:32.440071+0000","last_fullsized":"2024-05-10T21:06:32.440071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.800822+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.800822+0000","last_peered":"2024-05-10T21:06:25.800822+0000","last_clean":"2024-05-10T21:06:25.800822+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.800822+0000","last_undegraded":"2024-05-10T21:06:25.800822+0000","last_fullsized":"2024-05-10T21:06:25.800822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":263,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:35.266167+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:35.266167+0000","last_peered":"2024-05-10T21:08:35.266167+0000","last_clean":"2024-05-10T21:08:35.266167+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:35.266167+0000","last_undegraded":"2024-05-10T21:08:35.266167+0000","last_fullsized":"2024-05-10T21:08:35.266167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:29.888397+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:29.888397+0000","last_peered":"2024-05-10T21:06:29.888397+0000","last_clean":"2024-05-10T21:06:29.888397+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:29.888397+0000","last_undegraded":"2024-05-10T21:06:29.888397+0000","last_fullsized":"2024-05-10T21:06:29.888397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.641775+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:06:28.641775+0000","last_peered":"2024-05-10T21:06:28.641775+0000","last_clean":"2024-05-10T21:06:28.641775+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:06:28.641775+0000","last_undegraded":"2024-05-10T21:06:28.641775+0000","last_fullsized":"2024-05-10T21:06:28.641775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756012+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:31.756012+0000","last_peered":"2024-05-10T21:06:31.756012+0000","last_clean":"2024-05-10T21:06:31.756012+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:31.756012+0000","last_undegraded":"2024-05-10T21:06:31.756012+0000","last_fullsized":"2024-05-10T21:06:31.756012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:27.832050+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:06:27.832050+0000","last_peered":"2024-05-10T21:06:27.832050+0000","last_clean":"2024-05-10T21:06:27.832050+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:06:27.832050+0000","last_undegraded":"2024-05-10T21:06:27.832050+0000","last_fullsized":"2024-05-10T21:06:27.832050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756121+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:31.756121+0000","last_peered":"2024-05-10T21:06:31.756121+0000","last_clean":"2024-05-10T21:06:31.756121+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:31.756121+0000","last_undegraded":"2024-05-10T21:06:31.756121+0000","last_fullsized":"2024-05-10T21:06:31.756121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.918836+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:06:28.918836+0000","last_peered":"2024-05-10T21:06:28.918836+0000","last_clean":"2024-05-10T21:06:28.918836+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:06:28.918836+0000","last_undegraded":"2024-05-10T21:06:28.918836+0000","last_fullsized":"2024-05-10T21:06:28.918836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.331375+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:04:57.331375+0000","last_peered":"2024-05-10T21:04:57.331375+0000","last_clean":"2024-05-10T21:04:57.331375+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:04:57.331375+0000","last_undegraded":"2024-05-10T21:04:57.331375+0000","last_fullsized":"2024-05-10T21:04:57.331375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.160412+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:04:57.160412+0000","last_peered":"2024-05-10T21:04:57.160412+0000","last_clean":"2024-05-10T21:04:57.160412+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:04:57.160412+0000","last_undegraded":"2024-05-10T21:04:57.160412+0000","last_fullsized":"2024-05-10T21:04:57.160412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.548862+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:04:57.548862+0000","last_peered":"2024-05-10T21:04:57.548862+0000","last_clean":"2024-05-10T21:04:57.548862+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:04:57.548862+0000","last_undegraded":"2024-05-10T21:04:57.548862+0000","last_fullsized":"2024-05-10T21:04:57.548862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894315+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:04:56.894315+0000","last_peered":"2024-05-10T21:04:56.894315+0000","last_clean":"2024-05-10T21:04:56.894315+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:04:56.894315+0000","last_undegraded":"2024-05-10T21:04:56.894315+0000","last_fullsized":"2024-05-10T21:04:56.894315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878305+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:31.878305+0000","last_peered":"2024-05-10T21:06:31.878305+0000","last_clean":"2024-05-10T21:06:31.878305+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:31.878305+0000","last_undegraded":"2024-05-10T21:06:31.878305+0000","last_fullsized":"2024-05-10T21:06:31.878305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814417+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:30.814417+0000","last_peered":"2024-05-10T21:06:30.814417+0000","last_clean":"2024-05-10T21:06:30.814417+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:30.814417+0000","last_undegraded":"2024-05-10T21:06:30.814417+0000","last_fullsized":"2024-05-10T21:06:30.814417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814456+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:30.814456+0000","last_peered":"2024-05-10T21:06:30.814456+0000","last_clean":"2024-05-10T21:06:30.814456+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:30.814456+0000","last_undegraded":"2024-05-10T21:06:30.814456+0000","last_fullsized":"2024-05-10T21:06:30.814456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813238+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:28.813238+0000","last_peered":"2024-05-10T21:06:28.813238+0000","last_clean":"2024-05-10T21:06:28.813238+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:28.813238+0000","last_undegraded":"2024-05-10T21:06:28.813238+0000","last_fullsized":"2024-05-10T21:06:28.813238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74594,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1413,"num_read_kb":788,"num_write":181,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841610,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037839,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234070,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626529,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182513,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986286,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822759,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:41.061 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:08:41.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:40 smithi092 ceph-mon[34415]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:41.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:40 smithi092 ceph-mon[34415]: pgmap v454: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:41.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:40 smithi142 ceph-mon[34846]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:41.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:40 smithi142 ceph-mon[34846]: pgmap v454: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:41.674 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 -- 172.21.15.50:0/349011221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3078108740 msgr2=0x7f30781100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 --2- 172.21.15.50:0/349011221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3078108740 0x7f30781100a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f307000b0d0 tx=0x7f307000b3e0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 -- 172.21.15.50:0/349011221 shutdown_connections 2024-05-10T21:08:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 --2- 172.21.15.50:0/349011221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3078108740 0x7f30781100a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 --2- 172.21.15.50:0/349011221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30781034c0 0x7f30781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 --2- 172.21.15.50:0/349011221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f3078108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.742+0000 7f307d200700 1 -- 172.21.15.50:0/349011221 >> 172.21.15.50:0/349011221 conn(0x7f30780fedc0 msgr2=0x7f30781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.743+0000 7f307d200700 1 -- 172.21.15.50:0/349011221 shutdown_connections 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.743+0000 7f307d200700 1 -- 172.21.15.50:0/349011221 wait complete. 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.744+0000 7f307d200700 1 Processor -- start 2024-05-10T21:08:41.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.744+0000 7f307d200700 1 -- start start 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 0x7f30780ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3078108740 0x7f30780ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3078113370 con 0x7f30781034c0 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781131f0 con 0x7f3078107df0 2024-05-10T21:08:41.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f307d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30781134f0 con 0x7f3078108740 2024-05-10T21:08:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3075e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 0x7f30780ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3075400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3075400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54102/0 (socket says 172.21.15.50:54102) 2024-05-10T21:08:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3075400700 1 -- 172.21.15.50:0/2776565467 learned_addr learned my addr 172.21.15.50:0/2776565467 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:41.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3075e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 0x7f30780ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59170/0 (socket says 172.21.15.50:59170) 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.745+0000 7f3076800700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3078108740 0x7f30780ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 -- 172.21.15.50:0/2776565467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3078108740 msgr2=0x7f30780ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3078108740 0x7f30780ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 -- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 msgr2=0x7f30780ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 0x7f30780ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 -- 172.21.15.50:0/2776565467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781a9150 con 0x7f3078107df0 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f3075400700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f306400e970 tx=0x7f306400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f306ea00700 1 -- 172.21.15.50:0/2776565467 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064004a30 con 0x7f3078107df0 2024-05-10T21:08:41.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f306ea00700 1 -- 172.21.15.50:0/2776565467 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306400cc90 con 0x7f3078107df0 2024-05-10T21:08:41.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.746+0000 7f306ea00700 1 -- 172.21.15.50:0/2776565467 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064005610 con 0x7f3078107df0 2024-05-10T21:08:41.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.747+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781a9440 con 0x7f3078107df0 2024-05-10T21:08:41.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.747+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a9930 con 0x7f3078107df0 2024-05-10T21:08:41.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.753+0000 7f306ea00700 1 -- 172.21.15.50:0/2776565467 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f306400c7e0 con 0x7f3078107df0 2024-05-10T21:08:41.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.753+0000 7f308bd82b80 1 Processor -- start 2024-05-10T21:08:41.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.753+0000 7f308bd82b80 1 -- start start 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.753+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 0x563f24a192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f308bd82b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f24935a20 con 0x563f24895e40 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f308bd82b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563f248d3760 con 0x563f24a19830 2024-05-10T21:08:41.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f308bd82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f249319e0 con 0x563f24a139f0 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3076800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 0x563f24a192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3075e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3075e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54110/0 (socket says 172.21.15.50:54110) 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3075e00700 1 -- 172.21.15.50:0/603935918 learned_addr learned my addr 172.21.15.50:0/603935918 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3076800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 0x563f24a192f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59180/0 (socket says 172.21.15.50:59180) 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f3075400700 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.754+0000 7f306ea00700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 0x7f305c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 -- 172.21.15.50:0/603935918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 msgr2=0x563f24a13e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a13e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f306ea00700 1 -- 172.21.15.50:0/2776565467 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f30640e2280 con 0x7f3078107df0 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 -- 172.21.15.50:0/603935918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 msgr2=0x563f24a192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 0x563f24a192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 -- 172.21.15.50:0/603935918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f248828d0 con 0x563f24a19830 2024-05-10T21:08:41.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3076800700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 0x7f305c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.755+0000 7f3075e00700 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d80 tx=0x7f3068011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1e46743ece6a15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.756+0000 7f3063400700 1 -- 172.21.15.50:0/603935918 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068004230 con 0x563f24a19830 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.756+0000 7f3063400700 1 -- 172.21.15.50:0/603935918 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30680043d0 con 0x563f24a19830 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.756+0000 7f3063400700 1 -- 172.21.15.50:0/603935918 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068019830 con 0x563f24a19830 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 -- 172.21.15.50:0/603935918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 msgr2=0x563f24a134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d80 tx=0x7f3068011720 comp rx=0 tx=0).stop 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f3076800700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 0x7f305c0b4e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3070009fd0 tx=0x7f30700099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 -- 172.21.15.50:0/603935918 shutdown_connections 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a13e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24895e40 0x563f24a192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 --2- 172.21.15.50:0/603935918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a19830 0x563f24a134b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 -- 172.21.15.50:0/603935918 >> 172.21.15.50:0/603935918 conn(0x563f248daf70 msgr2=0x563f24a0e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 -- 172.21.15.50:0/603935918 shutdown_connections 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.757+0000 7f308bd82b80 1 -- 172.21.15.50:0/603935918 wait complete. 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.758+0000 7f308bd82b80 1 -- start start 2024-05-10T21:08:41.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.758+0000 7f308bd82b80 1 Processor -- start 2024-05-10T21:08:41.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f24a20310 con 0x563f24a19830 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563f2489bb20 con 0x563f24895e40 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f24a20490 con 0x563f24a139f0 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f308bd82b80 10 client.0 _ll_register_callbacks cb 0x563f248dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54124/0 (socket says 172.21.15.50:54124) 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075e00700 1 -- 172.21.15.50:0/18340921 learned_addr learned my addr 172.21.15.50:0/18340921 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075400700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3076800700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a1da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075e00700 1 -- 172.21.15.50:0/18340921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 msgr2=0x563f24a1d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.759+0000 7f3075e00700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a1d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3075e00700 1 -- 172.21.15.50:0/18340921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 msgr2=0x563f24a1da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3075e00700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a1da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3075e00700 1 -- 172.21.15.50:0/18340921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f248fb420 con 0x563f24895e40 2024-05-10T21:08:41.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3075e00700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3068000c00 tx=0x7f3068005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89204e042b44f15b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3057e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:41.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068027070 con 0x563f24895e40 2024-05-10T21:08:41.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f3068004950 con 0x563f24895e40 2024-05-10T21:08:41.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.760+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30680043d0 con 0x563f24895e40 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.761+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3068033050 con 0x563f24895e40 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.761+0000 7f308bd82b80 10 client.25148 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.761+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f24a2ae50 con 0x563f24895e40 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.761+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30680043d0 con 0x563f24895e40 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.761+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3068023460 con 0x563f24895e40 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 20 client.25148 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:41.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 4 client.25148 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:41.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f249e01c0 con 0x563f24a2d380 2024-05-10T21:08:41.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.762+0000 7f3075400700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.763+0000 7f3075400700 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a2f7a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6844a796459be68e server_cookie=3b33c05c987763 in_seq=0 out_seq=0 2024-05-10T21:08:41.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.763+0000 7f3057e00700 10 client.25148 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:41.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.763+0000 7f3057e00700 1 -- 172.21.15.50:0/18340921 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f249e01c0 con 0x563f24a2d380 2024-05-10T21:08:41.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.763+0000 7f3057e00700 10 client.25148 handle_command_reply: tid=0 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 client.25148 shutdown 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 msgr2=0x563f24a1cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3068000c00 tx=0x7f3068005320 comp rx=0 tx=0).stop 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 shutdown_connections 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a2f7a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a1cff0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f3057e00700 0 client.25148 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 --2- 172.21.15.50:0/18340921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a139f0 0x563f24a1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.764+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 >> 172.21.15.50:0/18340921 conn(0x563f248daf70 msgr2=0x563f24a124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.765+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 shutdown_connections 2024-05-10T21:08:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.765+0000 7f308bd82b80 1 -- 172.21.15.50:0/18340921 wait complete. 2024-05-10T21:08:41.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.765+0000 7f308bd82b80 20 client.25148 trim_cache size 0 max 16384 2024-05-10T21:08:41.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.788+0000 7f308bd82b80 1 Processor -- start 2024-05-10T21:08:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.788+0000 7f308bd82b80 1 -- start start 2024-05-10T21:08:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 0x563f24a1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30781b3680 con 0x563f24a139f0 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781b0f10 con 0x563f24895e40 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f308bd82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30781aed60 con 0x563f24a19830 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3075400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3076800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 0x563f24a1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3075400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59202/0 (socket says 172.21.15.50:59202) 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3076800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 0x563f24a1d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45426/0 (socket says 172.21.15.50:45426) 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3075400700 1 -- 172.21.15.50:0/4172741432 learned_addr learned my addr 172.21.15.50:0/4172741432 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:41.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.789+0000 7f3075e00700 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 -- 172.21.15.50:0/4172741432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 msgr2=0x563f24a1d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 0x563f24a1d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 -- 172.21.15.50:0/4172741432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 msgr2=0x563f24a134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 -- 172.21.15.50:0/4172741432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f24a0f260 con 0x563f24a139f0 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075e00700 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.790+0000 7f3075400700 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30640a95d0 tx=0x7f30640db040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec2663a5e82e435 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.791+0000 7f3062a00700 1 -- 172.21.15.50:0/4172741432 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640a9220 con 0x563f24a139f0 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.791+0000 7f3062a00700 1 -- 172.21.15.50:0/4172741432 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306400cad0 con 0x563f24a139f0 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.791+0000 7f3062a00700 1 -- 172.21.15.50:0/4172741432 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640a5060 con 0x563f24a139f0 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 -- 172.21.15.50:0/4172741432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 msgr2=0x563f24a08ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30640a95d0 tx=0x7f30640db040 comp rx=0 tx=0).stop 2024-05-10T21:08:41.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 -- 172.21.15.50:0/4172741432 shutdown_connections 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24895e40 0x563f24a134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24a19830 0x563f24a1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 --2- 172.21.15.50:0/4172741432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a139f0 0x563f24a08ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 -- 172.21.15.50:0/4172741432 >> 172.21.15.50:0/4172741432 conn(0x563f248daf70 msgr2=0x563f24a173f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.792+0000 7f308bd82b80 1 -- 172.21.15.50:0/4172741432 shutdown_connections 2024-05-10T21:08:41.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.793+0000 7f308bd82b80 1 -- 172.21.15.50:0/4172741432 wait complete. 2024-05-10T21:08:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.793+0000 7f308bd82b80 1 -- start start 2024-05-10T21:08:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.794+0000 7f308bd82b80 1 Processor -- start 2024-05-10T21:08:41.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.794+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.794+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.794+0000 7f308bd82b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f308bd82b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30781af3d0 con 0x563f24a19830 2024-05-10T21:08:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f308bd82b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781b0840 con 0x563f24a139f0 2024-05-10T21:08:41.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f308bd82b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30781afaa0 con 0x563f24895e40 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f308bd82b80 10 client.0 _ll_register_callbacks cb 0x563f248dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54144/0 (socket says 172.21.15.50:54144) 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45438/0 (socket says 172.21.15.50:45438) 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 -- 172.21.15.50:0/157279936 learned_addr learned my addr 172.21.15.50:0/157279936 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3076800700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 -- 172.21.15.50:0/157279936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 msgr2=0x563f24a34590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 -- 172.21.15.50:0/157279936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 msgr2=0x563f24a35010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a35010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075400700 1 -- 172.21.15.50:0/157279936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f24a969b0 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.795+0000 7f3075e00700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.796+0000 7f3075400700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30640e0120 tx=0x7f30640a9760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb19fdae42572196 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.796+0000 7f3057e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.796+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640a9040 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.796+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f30640cc050 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.796+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f24943400 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.797+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30640a98d0 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.797+0000 7f308bd82b80 10 client.25154 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.797+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f24a97b00 con 0x563f24a139f0 2024-05-10T21:08:41.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.797+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563f24943400 con 0x563f24a139f0 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.797+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f30640a9b30 con 0x563f24a139f0 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 10 client.25154 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 10 client.25154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 20 client.25154 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a9a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 4 client.25154 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:08:41.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f2493d4a0 con 0x563f24a2d380 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.798+0000 7f3075e00700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a9a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f3075e00700 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a9a110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850c86d5dea08e3b server_cookie=e7bc6fe969c095d6 in_seq=0 out_seq=0 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f3057e00700 10 client.25154 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f3057e00700 1 -- 172.21.15.50:0/157279936 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f2493d4a0 con 0x563f24a2d380 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f3057e00700 10 client.25154 handle_command_reply: tid=0 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f308bd82b80 1 client.25154 shutdown 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 msgr2=0x563f24a34ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.799+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30640e0120 tx=0x7f30640a9760 comp rx=0 tx=0).stop 2024-05-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 shutdown_connections 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f24a2d380 0x563f24a9a110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f24a19830 0x563f24a35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f24895e40 0x563f24a34590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f3057e00700 0 client.25154 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 --2- 172.21.15.50:0/157279936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f24a139f0 0x563f24a34ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 >> 172.21.15.50:0/157279936 conn(0x563f248daf70 msgr2=0x563f24a173f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 shutdown_connections 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 1 -- 172.21.15.50:0/157279936 wait complete. 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.800+0000 7f308bd82b80 20 client.25154 trim_cache size 0 max 16384 2024-05-10T21:08:41.803 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "ad893aa3-68dc-45e0-a690-d6b353a211d5": { 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "ad893aa3-68dc-45e0-a690-d6b353a211d5", 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:08:41.804 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:08:41.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 msgr2=0x7f305c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 0x7f305c0b4e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3070009fd0 tx=0x7f30700099e0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 msgr2=0x7f30780ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:41.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f306400e970 tx=0x7f306400ec80 comp rx=0 tx=0).stop 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 shutdown_connections 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f305c0b2900 0x7f305c0b4e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3078108740 0x7f30780ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30781034c0 0x7f30780ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 --2- 172.21.15.50:0/2776565467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078107df0 0x7f30780ff7e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.801+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 >> 172.21.15.50:0/2776565467 conn(0x7f30780fedc0 msgr2=0x7f307806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.802+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 shutdown_connections 2024-05-10T21:08:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:41.802+0000 7f307d200700 1 -- 172.21.15.50:0/2776565467 wait complete. 2024-05-10T21:08:43.112 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:42 smithi050 ceph-mon[29546]: pgmap v455: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:43.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:42 smithi092 ceph-mon[34415]: pgmap v455: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:43.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:42 smithi142 ceph-mon[34846]: pgmap v455: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:43.174 INFO:tasks.cephfs.filesystem:scrub status for tag:ad893aa3-68dc-45e0-a690-d6b353a211d5 - {'path': '/', 'tag': 'ad893aa3-68dc-45e0-a690-d6b353a211d5', 'options': 'recursive,force'} 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 -- 172.21.15.50:0/2316464990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 msgr2=0x7fee1c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 --2- 172.21.15.50:0/2316464990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c10b360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee08009920 tx=0x7fee08009c30 comp rx=0 tx=0).stop 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 -- 172.21.15.50:0/2316464990 shutdown_connections 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 --2- 172.21.15.50:0/2316464990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c10b8a0 0x7fee1c113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 --2- 172.21.15.50:0/2316464990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c106620 0x7fee1c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 --2- 172.21.15.50:0/2316464990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c10b360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 -- 172.21.15.50:0/2316464990 >> 172.21.15.50:0/2316464990 conn(0x7fee1c102030 msgr2=0x7fee1c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.710+0000 7fee21000700 1 -- 172.21.15.50:0/2316464990 shutdown_connections 2024-05-10T21:08:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.711+0000 7fee21000700 1 -- 172.21.15.50:0/2316464990 wait complete. 2024-05-10T21:08:44.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.712+0000 7fee21000700 1 Processor -- start 2024-05-10T21:08:44.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.712+0000 7fee21000700 1 -- start start 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c10b8a0 0x7fee1c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116500 con 0x7fee1c10b8a0 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116380 con 0x7fee1c106620 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee21000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c116680 con 0x7fee1c10af50 2024-05-10T21:08:44.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee1b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee1b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44786/0 (socket says 172.21.15.50:44786) 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee1b400700 1 -- 172.21.15.50:0/3920375019 learned_addr learned my addr 172.21.15.50:0/3920375019 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.713+0000 7fee1be00700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c10b8a0 0x7fee1c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 -- 172.21.15.50:0/3920375019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 msgr2=0x7fee1c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 -- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c10b8a0 msgr2=0x7fee1c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c10b8a0 0x7fee1c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 -- 172.21.15.50:0/3920375019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c1a1e00 con 0x7fee1c106620 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1aa00700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee1b400700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e9a0 tx=0x7fee0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:44.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00ca10 con 0x7fee1c106620 2024-05-10T21:08:44.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c1a20f0 con 0x7fee1c106620 2024-05-10T21:08:44.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee0c00cbb0 con 0x7fee1c106620 2024-05-10T21:08:44.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.714+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c018680 con 0x7fee1c106620 2024-05-10T21:08:44.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.715+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c1a9a10 con 0x7fee1c106620 2024-05-10T21:08:44.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.716+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c0512c0 con 0x7fee1c106620 2024-05-10T21:08:44.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.719+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fee0c010430 con 0x7fee1c106620 2024-05-10T21:08:44.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.720+0000 7fee13e00700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 0x7fee000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:44.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.721+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fee0c0e26d0 con 0x7fee1c106620 2024-05-10T21:08:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.721+0000 7fee1aa00700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 0x7fee000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.721+0000 7fee1aa00700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 0x7fee000b4d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee080098f0 tx=0x7fee08019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:44.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.722+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee0c0aa3c0 con 0x7fee1c106620 2024-05-10T21:08:44.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.965+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee1c063c00 con 0x7fee000b2880 2024-05-10T21:08:44.966 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:44 smithi050 ceph-mon[29546]: pgmap v456: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-10T21:08:44.966 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:44 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:08:44.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.976+0000 7fee13e00700 1 -- 172.21.15.50:0/3920375019 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258819 (secure 0 0 0) 0x7fee1c063c00 con 0x7fee000b2880 2024-05-10T21:08:44.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:44.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 msgr2=0x7fee000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:44.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 0x7fee000b4d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee080098f0 tx=0x7fee08019040 comp rx=0 tx=0).stop 2024-05-10T21:08:44.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 msgr2=0x7fee1c19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:44.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e9a0 tx=0x7fee0c00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 shutdown_connections 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fee000b2880 0x7fee000b4d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1c106620 0x7fee1c19cf30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee1c10b8a0 0x7fee1c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.983+0000 7fee21000700 1 --2- 172.21.15.50:0/3920375019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1c10af50 0x7fee1c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.984+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 >> 172.21.15.50:0/3920375019 conn(0x7fee1c102030 msgr2=0x7fee1c10c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.984+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 shutdown_connections 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:44.984+0000 7fee21000700 1 -- 172.21.15.50:0/3920375019 wait complete. 2024-05-10T21:08:44.985 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:45.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:44 smithi092 ceph-mon[34415]: pgmap v456: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-10T21:08:45.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:44 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:08:45.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:44 smithi142 ceph-mon[34846]: pgmap v456: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-10T21:08:45.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:44 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:08:45.849 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-05-10T21:08:43.685102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665196,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1525,"num_read_kb":868,"num_write":306,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005960"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878346+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:31.878346+0000","last_peered":"2024-05-10T21:06:31.878346+0000","last_clean":"2024-05-10T21:06:31.878346+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:31.878346+0000","last_undegraded":"2024-05-10T21:06:31.878346+0000","last_fullsized":"2024-05-10T21:06:31.878346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813313+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:28.813313+0000","last_peered":"2024-05-10T21:06:28.813313+0000","last_clean":"2024-05-10T21:06:28.813313+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:28.813313+0000","last_undegraded":"2024-05-10T21:06:28.813313+0000","last_fullsized":"2024-05-10T21:06:28.813313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813290+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:28.813290+0000","last_peered":"2024-05-10T21:06:28.813290+0000","last_clean":"2024-05-10T21:06:28.813290+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:28.813290+0000","last_undegraded":"2024-05-10T21:06:28.813290+0000","last_fullsized":"2024-05-10T21:06:28.813290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566835+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:25.566835+0000","last_peered":"2024-05-10T21:06:25.566835+0000","last_clean":"2024-05-10T21:06:25.566835+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:25.566835+0000","last_undegraded":"2024-05-10T21:06:25.566835+0000","last_fullsized":"2024-05-10T21:06:25.566835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:28.939887+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:08:28.939887+0000","last_peered":"2024-05-10T21:08:28.939887+0000","last_clean":"2024-05-10T21:08:28.939887+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:08:28.939887+0000","last_undegraded":"2024-05-10T21:08:28.939887+0000","last_fullsized":"2024-05-10T21:08:28.939887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572865+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.572865+0000","last_peered":"2024-05-10T21:06:25.572865+0000","last_clean":"2024-05-10T21:06:25.572865+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.572865+0000","last_undegraded":"2024-05-10T21:06:25.572865+0000","last_fullsized":"2024-05-10T21:06:25.572865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813423+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:28.813423+0000","last_peered":"2024-05-10T21:06:28.813423+0000","last_clean":"2024-05-10T21:06:28.813423+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:28.813423+0000","last_undegraded":"2024-05-10T21:06:28.813423+0000","last_fullsized":"2024-05-10T21:06:28.813423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":270,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:39.118850+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:39.118850+0000","last_peered":"2024-05-10T21:08:39.118850+0000","last_clean":"2024-05-10T21:08:39.118850+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:39.118850+0000","last_undegraded":"2024-05-10T21:08:39.118850+0000","last_fullsized":"2024-05-10T21:08:39.118850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":267,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:41.121427+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:41.121427+0000","last_peered":"2024-05-10T21:08:41.121427+0000","last_clean":"2024-05-10T21:08:41.121427+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:41.121427+0000","last_undegraded":"2024-05-10T21:08:41.121427+0000","last_fullsized":"2024-05-10T21:08:41.121427+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":551,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:42.122665+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:42.122665+0000","last_peered":"2024-05-10T21:08:42.122665+0000","last_clean":"2024-05-10T21:08:42.122665+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:42.122665+0000","last_undegraded":"2024-05-10T21:08:42.122665+0000","last_fullsized":"2024-05-10T21:08:42.122665+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70142,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":313,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":273,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:42.284514+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:42.284514+0000","last_peered":"2024-05-10T21:08:42.284514+0000","last_clean":"2024-05-10T21:08:42.284514+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:42.284514+0000","last_undegraded":"2024-05-10T21:08:42.284514+0000","last_fullsized":"2024-05-10T21:08:42.284514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:23.917882+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:06:23.917882+0000","last_peered":"2024-05-10T21:06:23.917882+0000","last_clean":"2024-05-10T21:06:23.917882+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:06:23.917882+0000","last_undegraded":"2024-05-10T21:06:23.917882+0000","last_fullsized":"2024-05-10T21:06:23.917882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.440092+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:06:32.440092+0000","last_peered":"2024-05-10T21:06:32.440092+0000","last_clean":"2024-05-10T21:06:32.440092+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:06:32.440092+0000","last_undegraded":"2024-05-10T21:06:32.440092+0000","last_fullsized":"2024-05-10T21:06:32.440092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:24.928701+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:06:24.928701+0000","last_peered":"2024-05-10T21:06:24.928701+0000","last_clean":"2024-05-10T21:06:24.928701+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:06:24.928701+0000","last_undegraded":"2024-05-10T21:06:24.928701+0000","last_fullsized":"2024-05-10T21:06:24.928701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.440071+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:32.440071+0000","last_peered":"2024-05-10T21:06:32.440071+0000","last_clean":"2024-05-10T21:06:32.440071+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:32.440071+0000","last_undegraded":"2024-05-10T21:06:32.440071+0000","last_fullsized":"2024-05-10T21:06:32.440071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.800822+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.800822+0000","last_peered":"2024-05-10T21:06:25.800822+0000","last_clean":"2024-05-10T21:06:25.800822+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.800822+0000","last_undegraded":"2024-05-10T21:06:25.800822+0000","last_fullsized":"2024-05-10T21:06:25.800822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":268,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:40.278733+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:40.278733+0000","last_peered":"2024-05-10T21:08:40.278733+0000","last_clean":"2024-05-10T21:08:40.278733+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:40.278733+0000","last_undegraded":"2024-05-10T21:08:40.278733+0000","last_fullsized":"2024-05-10T21:08:40.278733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:29.888397+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:29.888397+0000","last_peered":"2024-05-10T21:06:29.888397+0000","last_clean":"2024-05-10T21:06:29.888397+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:29.888397+0000","last_undegraded":"2024-05-10T21:06:29.888397+0000","last_fullsized":"2024-05-10T21:06:29.888397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.641775+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:06:28.641775+0000","last_peered":"2024-05-10T21:06:28.641775+0000","last_clean":"2024-05-10T21:06:28.641775+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:06:28.641775+0000","last_undegraded":"2024-05-10T21:06:28.641775+0000","last_fullsized":"2024-05-10T21:06:28.641775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756012+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:31.756012+0000","last_peered":"2024-05-10T21:06:31.756012+0000","last_clean":"2024-05-10T21:06:31.756012+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:31.756012+0000","last_undegraded":"2024-05-10T21:06:31.756012+0000","last_fullsized":"2024-05-10T21:06:31.756012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:27.832050+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:06:27.832050+0000","last_peered":"2024-05-10T21:06:27.832050+0000","last_clean":"2024-05-10T21:06:27.832050+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:06:27.832050+0000","last_undegraded":"2024-05-10T21:06:27.832050+0000","last_fullsized":"2024-05-10T21:06:27.832050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756121+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:31.756121+0000","last_peered":"2024-05-10T21:06:31.756121+0000","last_clean":"2024-05-10T21:06:31.756121+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:31.756121+0000","last_undegraded":"2024-05-10T21:06:31.756121+0000","last_fullsized":"2024-05-10T21:06:31.756121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:16.983580+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:08:16.983580+0000","last_peered":"2024-05-10T21:08:16.983580+0000","last_clean":"2024-05-10T21:08:16.983580+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:08:16.983580+0000","last_undegraded":"2024-05-10T21:08:16.983580+0000","last_fullsized":"2024-05-10T21:08:16.983580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570200+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:25.570200+0000","last_peered":"2024-05-10T21:06:25.570200+0000","last_clean":"2024-05-10T21:06:25.570200+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:25.570200+0000","last_undegraded":"2024-05-10T21:06:25.570200+0000","last_fullsized":"2024-05-10T21:06:25.570200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:33.642493+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:06:33.642493+0000","last_peered":"2024-05-10T21:06:33.642493+0000","last_clean":"2024-05-10T21:06:33.642493+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:06:33.642493+0000","last_undegraded":"2024-05-10T21:06:33.642493+0000","last_fullsized":"2024-05-10T21:06:33.642493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581613+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:25.581613+0000","last_peered":"2024-05-10T21:06:25.581613+0000","last_clean":"2024-05-10T21:06:25.581613+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:25.581613+0000","last_undegraded":"2024-05-10T21:06:25.581613+0000","last_fullsized":"2024-05-10T21:06:25.581613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.154277+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:06:32.154277+0000","last_peered":"2024-05-10T21:06:32.154277+0000","last_clean":"2024-05-10T21:06:32.154277+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:06:32.154277+0000","last_undegraded":"2024-05-10T21:06:32.154277+0000","last_fullsized":"2024-05-10T21:06:32.154277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:29.888329+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:06:29.888329+0000","last_peered":"2024-05-10T21:06:29.888329+0000","last_clean":"2024-05-10T21:06:29.888329+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:06:29.888329+0000","last_undegraded":"2024-05-10T21:06:29.888329+0000","last_fullsized":"2024-05-10T21:06:29.888329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579185+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:25.579185+0000","last_peered":"2024-05-10T21:06:25.579185+0000","last_clean":"2024-05-10T21:06:25.579185+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:25.579185+0000","last_undegraded":"2024-05-10T21:06:25.579185+0000","last_fullsized":"2024-05-10T21:06:25.579185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814495+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:06:30.814495+0000","last_peered":"2024-05-10T21:06:30.814495+0000","last_clean":"2024-05-10T21:06:30.814495+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:06:30.814495+0000","last_undegraded":"2024-05-10T21:06:30.814495+0000","last_fullsized":"2024-05-10T21:06:30.814495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566631+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:25.566631+0000","last_peered":"2024-05-10T21:06:25.566631+0000","last_clean":"2024-05-10T21:06:25.566631+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:25.566631+0000","last_undegraded":"2024-05-10T21:06:25.566631+0000","last_fullsized":"2024-05-10T21:06:25.566631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878305+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:31.878305+0000","last_peered":"2024-05-10T21:06:31.878305+0000","last_clean":"2024-05-10T21:06:31.878305+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:31.878305+0000","last_undegraded":"2024-05-10T21:06:31.878305+0000","last_fullsized":"2024-05-10T21:06:31.878305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814417+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:30.814417+0000","last_peered":"2024-05-10T21:06:30.814417+0000","last_clean":"2024-05-10T21:06:30.814417+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:30.814417+0000","last_undegraded":"2024-05-10T21:06:30.814417+0000","last_fullsized":"2024-05-10T21:06:30.814417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814456+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:30.814456+0000","last_peered":"2024-05-10T21:06:30.814456+0000","last_clean":"2024-05-10T21:06:30.814456+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:30.814456+0000","last_undegraded":"2024-05-10T21:06:30.814456+0000","last_fullsized":"2024-05-10T21:06:30.814456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813238+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:28.813238+0000","last_peered":"2024-05-10T21:06:28.813238+0000","last_clean":"2024-05-10T21:06:28.813238+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:28.813238+0000","last_undegraded":"2024-05-10T21:06:28.813238+0000","last_fullsized":"2024-05-10T21:06:28.813238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576347+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:25.576347+0000","last_peered":"2024-05-10T21:06:25.576347+0000","last_clean":"2024-05-10T21:06:25.576347+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:25.576347+0000","last_undegraded":"2024-05-10T21:06:25.576347+0000","last_fullsized":"2024-05-10T21:06:25.576347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.551328+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:25.551328+0000","last_peered":"2024-05-10T21:06:25.551328+0000","last_clean":"2024-05-10T21:06:25.551328+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:25.551328+0000","last_undegraded":"2024-05-10T21:06:25.551328+0000","last_fullsized":"2024-05-10T21:06:25.551328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.557329+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:25.557329+0000","last_peered":"2024-05-10T21:06:25.557329+0000","last_clean":"2024-05-10T21:06:25.557329+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:25.557329+0000","last_undegraded":"2024-05-10T21:06:25.557329+0000","last_fullsized":"2024-05-10T21:06:25.557329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571761+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:25.571761+0000","last_peered":"2024-05-10T21:06:25.571761+0000","last_clean":"2024-05-10T21:06:25.571761+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:25.571761+0000","last_undegraded":"2024-05-10T21:06:25.571761+0000","last_fullsized":"2024-05-10T21:06:25.571761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.570007+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:25.570007+0000","last_peered":"2024-05-10T21:06:25.570007+0000","last_clean":"2024-05-10T21:06:25.570007+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:25.570007+0000","last_undegraded":"2024-05-10T21:06:25.570007+0000","last_fullsized":"2024-05-10T21:06:25.570007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74594,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1449,"num_read_kb":804,"num_write":181,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645384,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841611,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037840,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234072,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430302,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626530,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182514,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822760,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:45.853 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T21:08:45.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T21:08:46.341 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:46 smithi092 ceph-mon[34415]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:46 smithi092 ceph-mon[34415]: pgmap v457: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:08:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:08:47.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:08:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:46 smithi142 ceph-mon[34846]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:46 smithi142 ceph-mon[34846]: pgmap v457: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:08:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:08:47.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:08:47.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:46 smithi050 ceph-mon[29546]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:47.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:46 smithi050 ceph-mon[29546]: pgmap v457: 97 pgs: 97 active+clean; 650 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:08:47.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:08:47.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:08:47.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:08:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 -- 172.21.15.50:0/1321844271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c105d20 msgr2=0x7f490c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 --2- 172.21.15.50:0/1321844271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c105d20 0x7f490c106a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009900 tx=0x7f48fc02a400 comp rx=0 tx=0).stop 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 -- 172.21.15.50:0/1321844271 shutdown_connections 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 --2- 172.21.15.50:0/1321844271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 0x7f490c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 --2- 172.21.15.50:0/1321844271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c106f60 0x7f490c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 --2- 172.21.15.50:0/1321844271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c105d20 0x7f490c106a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 -- 172.21.15.50:0/1321844271 >> 172.21.15.50:0/1321844271 conn(0x7f490c100f10 msgr2=0x7f490c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 -- 172.21.15.50:0/1321844271 shutdown_connections 2024-05-10T21:08:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.227+0000 7f4910c00640 1 -- 172.21.15.50:0/1321844271 wait complete. 2024-05-10T21:08:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.229+0000 7f4910c00640 1 Processor -- start 2024-05-10T21:08:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.229+0000 7f4910c00640 1 -- start start 2024-05-10T21:08:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.229+0000 7f4910c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.229+0000 7f4910c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c106f60 0x7f490c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f4910c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 0x7f490c1a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f4910c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1117b0 con 0x7f490c105d20 2024-05-10T21:08:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f4910c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111630 con 0x7f490c10bc60 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f4910c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111930 con 0x7f490c106f60 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48362/0 (socket says 172.21.15.50:48362) 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 -- 172.21.15.50:0/3178632111 learned_addr learned my addr 172.21.15.50:0/3178632111 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f4902a00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c106f60 0x7f490c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490be00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 0x7f490c1a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 -- 172.21.15.50:0/3178632111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c106f60 msgr2=0x7f490c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c106f60 0x7f490c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 -- 172.21.15.50:0/3178632111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 msgr2=0x7f490c1a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 0x7f490c1a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.230+0000 7f490b400640 1 -- 172.21.15.50:0/3178632111 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a3f10 con 0x7f490c105d20 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f490b400640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48fc006010 tx=0x7f48fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f4908c00640 1 -- 172.21.15.50:0/3178632111 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03f030 con 0x7f490c105d20 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f4908c00640 1 -- 172.21.15.50:0/3178632111 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc031910 con 0x7f490c105d20 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f4908c00640 1 -- 172.21.15.50:0/3178632111 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc035070 con 0x7f490c105d20 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a41a0 con 0x7f490c105d20 2024-05-10T21:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.231+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a4640 con 0x7f490c105d20 2024-05-10T21:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 Processor -- start 2024-05-10T21:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 -- start start 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56348260c550 con 0x5634823061b0 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.235+0000 7f49191c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5634825b1bc0 con 0x5634822dd2f0 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f49191c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563482608510 con 0x5634822e54d0 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f4908c00640 1 -- 172.21.15.50:0/3178632111 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f48fc042030 con 0x7f490c105d20 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48364/0 (socket says 172.21.15.50:48364) 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 -- 172.21.15.50:0/4169139382 learned_addr learned my addr 172.21.15.50:0/4169139382 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f4902a00640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 -- 172.21.15.50:0/4169139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 msgr2=0x5634826af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490b400640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 -- 172.21.15.50:0/4169139382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 msgr2=0x5634826a8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826a8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490be00640 1 -- 172.21.15.50:0/4169139382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348231e210 con 0x5634823061b0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.236+0000 7f490b400640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.237+0000 7f490be00640 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4904009ca0 tx=0x7f4904011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9ca1d7cc1963f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.237+0000 7f48fbe00640 1 -- 172.21.15.50:0/4169139382 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401e070 con 0x5634823061b0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.237+0000 7f48fbe00640 1 -- 172.21.15.50:0/4169139382 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4904009e50 con 0x5634823061b0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.237+0000 7f48fbe00640 1 -- 172.21.15.50:0/4169139382 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490400ea50 con 0x5634823061b0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f4908c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 0x7f48f00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f4908c00640 1 -- 172.21.15.50:0/3178632111 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f48fc1024a0 con 0x7f490c105d20 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f490b400640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 0x7f48f00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 -- 172.21.15.50:0/4169139382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 msgr2=0x5634826a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4904009ca0 tx=0x7f4904011810 comp rx=0 tx=0).stop 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 -- 172.21.15.50:0/4169139382 shutdown_connections 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 --2- 172.21.15.50:0/4169139382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x5634826a8300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.238+0000 7f49191c7740 1 -- 172.21.15.50:0/4169139382 >> 172.21.15.50:0/4169139382 conn(0x5634822fa300 msgr2=0x5634826a3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.239+0000 7f49191c7740 1 -- 172.21.15.50:0/4169139382 shutdown_connections 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.239+0000 7f490b400640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 0x7f48f00b4d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5634826a96e0 tx=0x7f48fc0ec040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.239+0000 7f49191c7740 1 -- 172.21.15.50:0/4169139382 wait complete. 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.239+0000 7f49191c7740 1 -- start start 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 Processor -- start 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 0x5634826b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 0x5634826b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634823061b0 0x5634826b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5634826b01d0 con 0x5634822e54d0 2024-05-10T21:08:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5634825c5660 con 0x5634823061b0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5634826b0350 con 0x5634822dd2f0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f49191c7740 10 client.0 _ll_register_callbacks cb 0x56348258f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f490be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 0x5634826b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f4902a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 0x5634826b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.240+0000 7f490be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 0x5634826b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54410/0 (socket says 172.21.15.50:54410) 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f490be00640 1 -- 172.21.15.50:0/2029045308 learned_addr learned my addr 172.21.15.50:0/2029045308 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f490be00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634823061b0 0x5634826b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 -- 172.21.15.50:0/2029045308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 msgr2=0x5634826b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 0x5634826b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 -- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634823061b0 msgr2=0x5634826b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634823061b0 0x5634826b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 -- 172.21.15.50:0/2029045308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634823ea620 con 0x5634822e54d0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f4902a00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 0x5634826b2340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f490c19d470 tx=0x7f48f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe44e7b2c2854fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.241+0000 7f48f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4009e50 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f48f4004b20 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4904009e50 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48f4022070 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f49191c7740 10 client.15051 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634826bc210 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.242+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4904009e50 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f48f401b470 con 0x5634822e54d0 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 10 client.15051 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 10 client.15051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 20 client.15051 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5634822c4d80 0x5634826bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 4 client.15051 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56348261a160 con 0x5634822c4d80 2024-05-10T21:08:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.243+0000 7f490be00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5634822c4d80 0x5634826bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.244+0000 7f490be00640 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5634822c4d80 0x5634826bf910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753494cc49596915 server_cookie=45e9ce315a19fb79 in_seq=0 out_seq=0 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.244+0000 7f48f8c00640 10 client.15051 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f48f8c00640 1 -- 172.21.15.50:0/2029045308 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56348261a160 con 0x5634822c4d80 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f48f8c00640 10 client.15051 handle_command_reply: tid=0 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 client.15051 shutdown 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 msgr2=0x5634826b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 0x5634826b2340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f490c19d470 tx=0x7f48f400f790 comp rx=0 tx=0).stop 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 shutdown_connections 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5634822c4d80 0x5634826bf910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634823061b0 0x5634826b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822e54d0 0x5634826b2340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 --2- 172.21.15.50:0/2029045308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822dd2f0 0x5634826b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f48f8c00640 0 client.15051 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 >> 172.21.15.50:0/2029045308 conn(0x5634822fa300 msgr2=0x5634826a2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 shutdown_connections 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 1 -- 172.21.15.50:0/2029045308 wait complete. 2024-05-10T21:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.245+0000 7f49191c7740 20 client.15051 trim_cache size 0 max 16384 2024-05-10T21:08:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.262+0000 7f49191c7740 1 Processor -- start 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.262+0000 7f49191c7740 1 -- start start 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822dd2f0 0x5634826c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822e54d0 0x5634826b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1afba0 con 0x5634822dd2f0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1ad470 con 0x5634822e54d0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f49191c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1ab2c0 con 0x5634823061b0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f490be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822dd2f0 0x5634826c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f490be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f490be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54422/0 (socket says 172.21.15.50:54422) 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f490be00640 1 -- 172.21.15.50:0/2455877790 learned_addr learned my addr 172.21.15.50:0/2455877790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.263+0000 7f4902a00640 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822e54d0 0x5634826b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 -- 172.21.15.50:0/2455877790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822e54d0 msgr2=0x5634826b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822e54d0 0x5634826b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 -- 172.21.15.50:0/2455877790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822dd2f0 msgr2=0x5634826c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822dd2f0 0x5634826c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 -- 172.21.15.50:0/2455877790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348269f4a0 con 0x5634823061b0 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f490be00640 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4904009c40 tx=0x7f4904005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae2ff43c63da4eda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f48fb400640 1 -- 172.21.15.50:0/2455877790 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401e040 con 0x5634823061b0 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f48fb400640 1 -- 172.21.15.50:0/2455877790 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490400ee30 con 0x5634823061b0 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.264+0000 7f48fb400640 1 -- 172.21.15.50:0/2455877790 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904003c00 con 0x5634823061b0 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 -- 172.21.15.50:0/2455877790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 msgr2=0x5634826b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4904009c40 tx=0x7f4904005f90 comp rx=0 tx=0).stop 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 -- 172.21.15.50:0/2455877790 shutdown_connections 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634823061b0 0x5634826b9700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822e54d0 0x5634826b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 --2- 172.21.15.50:0/2455877790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634822dd2f0 0x5634826c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 -- 172.21.15.50:0/2455877790 >> 172.21.15.50:0/2455877790 conn(0x5634822fa300 msgr2=0x5634826ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 -- 172.21.15.50:0/2455877790 shutdown_connections 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.265+0000 7f49191c7740 1 -- 172.21.15.50:0/2455877790 wait complete. 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 -- start start 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 Processor -- start 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1ab930 con 0x5634823061b0 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1acda0 con 0x5634822dd2f0 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1ac000 con 0x5634822e54d0 2024-05-10T21:08:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.266+0000 7f49191c7740 10 client.0 _ll_register_callbacks cb 0x7f48f4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f490be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f490be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f490be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48384/0 (socket says 172.21.15.50:48384) 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f490be00640 1 -- 172.21.15.50:0/3398907362 learned_addr learned my addr 172.21.15.50:0/3398907362 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 -- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 msgr2=0x5634826b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 -- 172.21.15.50:0/3398907362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 msgr2=0x56348272e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 -- 172.21.15.50:0/3398907362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348272ea30 con 0x5634822e54d0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f490be00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f4902a00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826b6fc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48f4002230 tx=0x7f48f401b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b65b049b49b939e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.267+0000 7f48f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:08:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f401b6a0 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f48f4017070 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490400ee30 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48f401bd10 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f49191c7740 10 client.25171 fetch_fsmap learned FSMap version 25 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56348272fcd0 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.268+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490400ee30 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f48f40279f0 con 0x5634822e54d0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 10 client.25171 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 10 client.25171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 20 client.25171 populate_metadata read hostname 'smithi050' 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56348229a1c0 0x563482732740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 4 client.25171 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563482613fd0 con 0x56348229a1c0 2024-05-10T21:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.269+0000 7f490be00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56348229a1c0 0x563482732740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.270+0000 7f490be00640 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56348229a1c0 0x563482732740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a292fa9c34bcf90b server_cookie=9502ba8a78412fed in_seq=0 out_seq=0 2024-05-10T21:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.270+0000 7f48f8c00640 10 client.25171 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.282+0000 7f48f8c00640 1 -- 172.21.15.50:0/3398907362 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563482613fd0 con 0x56348229a1c0 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.282+0000 7f48f8c00640 10 client.25171 handle_command_reply: tid=0 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.282+0000 7f49191c7740 1 client.25171 shutdown 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.282+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 msgr2=0x5634826b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.282+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826b6fc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48f4002230 tx=0x7f48f401b470 comp rx=0 tx=0).stop 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 shutdown_connections 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56348229a1c0 0x563482732740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634822e54d0 0x5634826b6fc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634823061b0 0x56348272e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 --2- 172.21.15.50:0/3398907362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5634822dd2f0 0x5634826b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f48f8c00640 0 client.25171 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:08:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 >> 172.21.15.50:0/3398907362 conn(0x5634822fa300 msgr2=0x5634826a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 shutdown_connections 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 1 -- 172.21.15.50:0/3398907362 wait complete. 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.283+0000 7f49191c7740 20 client.25171 trim_cache size 0 max 16384 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:08:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 msgr2=0x7f48f00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 0x7f48f00b4d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5634826a96e0 tx=0x7f48fc0ec040 comp rx=0 tx=0).stop 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 msgr2=0x7f490c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48fc006010 tx=0x7f48fc004270 comp rx=0 tx=0).stop 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 shutdown_connections 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f48f00b2860 0x7f48f00b4d70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f490c106f60 0x7f490c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f490c10bc60 0x7f490c1a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 --2- 172.21.15.50:0/3178632111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f490c105d20 0x7f490c19c3d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 >> 172.21.15.50:0/3178632111 conn(0x7f490c100f10 msgr2=0x7f490c107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 shutdown_connections 2024-05-10T21:08:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:08:47.284+0000 7f4910c00640 1 -- 172.21.15.50:0/3178632111 wait complete. 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 -- 172.21.15.50:0/2575486892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010af50 msgr2=0x7f918010b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 --2- 172.21.15.50:0/2575486892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010af50 0x7f918010b360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f916c009990 tx=0x7f916c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 -- 172.21.15.50:0/2575486892 shutdown_connections 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 --2- 172.21.15.50:0/2575486892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f918010b8a0 0x7f9180113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 --2- 172.21.15.50:0/2575486892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9180106620 0x7f9180106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 --2- 172.21.15.50:0/2575486892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010af50 0x7f918010b360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 -- 172.21.15.50:0/2575486892 >> 172.21.15.50:0/2575486892 conn(0x7f9180102030 msgr2=0x7f9180104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 -- 172.21.15.50:0/2575486892 shutdown_connections 2024-05-10T21:08:48.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.550+0000 7f9186e00700 1 -- 172.21.15.50:0/2575486892 wait complete. 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.552+0000 7f9186e00700 1 Processor -- start 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.552+0000 7f9186e00700 1 -- start start 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.552+0000 7f9186e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 0x7f918019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.552+0000 7f9186e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9186e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010b8a0 0x7f91801a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9186e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91801164b0 con 0x7f9180106620 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9186e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9180116330 con 0x7f918010b8a0 2024-05-10T21:08:48.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9186e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9180116630 con 0x7f918010af50 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9185a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 0x7f918019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9185000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9185a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 0x7f918019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48398/0 (socket says 172.21.15.50:48398) 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9185a00700 1 -- 172.21.15.50:0/3279392656 learned_addr learned my addr 172.21.15.50:0/3279392656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9185000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54460/0 (socket says 172.21.15.50:54460) 2024-05-10T21:08:48.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.553+0000 7f9186400700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010b8a0 0x7f91801a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 -- 172.21.15.50:0/3279392656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010b8a0 msgr2=0x7f91801a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010b8a0 0x7f91801a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 -- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 msgr2=0x7f918019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 0x7f918019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 -- 172.21.15.50:0/3279392656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801a1d30 con 0x7f918010af50 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9185000700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f916c0059d0 tx=0x7f916c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c027070 con 0x7f918010af50 2024-05-10T21:08:48.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916c00b610 con 0x7f918010af50 2024-05-10T21:08:48.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91801a1fc0 con 0x7f918010af50 2024-05-10T21:08:48.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.554+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c022dd0 con 0x7f918010af50 2024-05-10T21:08:48.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.555+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a2470 con 0x7f918010af50 2024-05-10T21:08:48.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.556+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91800512c0 con 0x7f918010af50 2024-05-10T21:08:48.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.558+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f916c02a070 con 0x7f918010af50 2024-05-10T21:08:48.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.560+0000 7f917e000700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 0x7f91680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:48.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.560+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f916c0e3460 con 0x7f918010af50 2024-05-10T21:08:48.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.560+0000 7f9185a00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 0x7f91680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:48.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.561+0000 7f9185a00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 0x7f91680b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d20 tx=0x7f9174009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:48.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.562+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f916c013070 con 0x7f918010af50 2024-05-10T21:08:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.803+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 --> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f918019e1c0 con 0x7f91680b2880 2024-05-10T21:08:48.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.825+0000 7f917e000700 1 -- 172.21.15.50:0/3279392656 <== mgr.14152 v2:172.21.15.50:6800/628586267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258823 (secure 0 0 0) 0x7f918019e1c0 con 0x7f91680b2880 2024-05-10T21:08:48.829 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:48.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.831+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 msgr2=0x7f91680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:48.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.831+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 0x7f91680b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d20 tx=0x7f9174009260 comp rx=0 tx=0).stop 2024-05-10T21:08:48.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.831+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 msgr2=0x7f918019d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:48.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.831+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f916c0059d0 tx=0x7f916c0057c0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 shutdown_connections 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91680b2880 0x7f91680b4d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918010b8a0 0x7f91801a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9180106620 0x7f918019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 --2- 172.21.15.50:0/3279392656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918010af50 0x7f918019d400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 >> 172.21.15.50:0/3279392656 conn(0x7f9180102030 msgr2=0x7f918010a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 shutdown_connections 2024-05-10T21:08:48.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:48.832+0000 7f9186e00700 1 -- 172.21.15.50:0/3279392656 wait complete. 2024-05-10T21:08:48.834 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-10T21:08:49.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:48 smithi092 ceph-mon[34415]: pgmap v458: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T21:08:49.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:48 smithi142 ceph-mon[34846]: pgmap v458: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T21:08:49.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:48 smithi050 ceph-mon[29546]: pgmap v458: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T21:08:49.748 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-05-10T21:08:47.687432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668076,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":1550,"num_read_kb":883,"num_write":317,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":757,"ondisk_log_size":757,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374196,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567628,"statfs":{"total":1151940427776,"available":1151557251072,"internally_reserved":0,"allocated":12865536,"data_stored":7232904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2880,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006179"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878346+0000","last_change":"2024-05-10T21:04:23.927061+0000","last_active":"2024-05-10T21:06:31.878346+0000","last_peered":"2024-05-10T21:06:31.878346+0000","last_clean":"2024-05-10T21:06:31.878346+0000","last_became_active":"2024-05-10T21:04:23.926644+0000","last_became_peered":"2024-05-10T21:04:23.926644+0000","last_unstale":"2024-05-10T21:06:31.878346+0000","last_undegraded":"2024-05-10T21:06:31.878346+0000","last_fullsized":"2024-05-10T21:06:31.878346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:37:26.207616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813313+0000","last_change":"2024-05-10T21:04:23.919537+0000","last_active":"2024-05-10T21:06:28.813313+0000","last_peered":"2024-05-10T21:06:28.813313+0000","last_clean":"2024-05-10T21:06:28.813313+0000","last_became_active":"2024-05-10T21:04:23.919162+0000","last_became_peered":"2024-05-10T21:04:23.919162+0000","last_unstale":"2024-05-10T21:06:28.813313+0000","last_undegraded":"2024-05-10T21:06:28.813313+0000","last_fullsized":"2024-05-10T21:06:28.813313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:03:06.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813290+0000","last_change":"2024-05-10T21:04:23.929908+0000","last_active":"2024-05-10T21:06:28.813290+0000","last_peered":"2024-05-10T21:06:28.813290+0000","last_clean":"2024-05-10T21:06:28.813290+0000","last_became_active":"2024-05-10T21:04:23.923344+0000","last_became_peered":"2024-05-10T21:04:23.923344+0000","last_unstale":"2024-05-10T21:06:28.813290+0000","last_undegraded":"2024-05-10T21:06:28.813290+0000","last_fullsized":"2024-05-10T21:06:28.813290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:03:00.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556763+0000","last_change":"2024-05-10T21:04:23.918154+0000","last_active":"2024-05-10T21:06:25.556763+0000","last_peered":"2024-05-10T21:06:25.556763+0000","last_clean":"2024-05-10T21:06:25.556763+0000","last_became_active":"2024-05-10T21:04:23.917218+0000","last_became_peered":"2024-05-10T21:04:23.917218+0000","last_unstale":"2024-05-10T21:06:25.556763+0000","last_undegraded":"2024-05-10T21:06:25.556763+0000","last_fullsized":"2024-05-10T21:06:25.556763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:42:30.791594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568305+0000","last_change":"2024-05-10T21:04:23.940144+0000","last_active":"2024-05-10T21:06:25.568305+0000","last_peered":"2024-05-10T21:06:25.568305+0000","last_clean":"2024-05-10T21:06:25.568305+0000","last_became_active":"2024-05-10T21:04:23.939716+0000","last_became_peered":"2024-05-10T21:04:23.939716+0000","last_unstale":"2024-05-10T21:06:25.568305+0000","last_undegraded":"2024-05-10T21:06:25.568305+0000","last_fullsized":"2024-05-10T21:06:25.568305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:31.931200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.562542+0000","last_change":"2024-05-10T21:04:23.929568+0000","last_active":"2024-05-10T21:06:25.562542+0000","last_peered":"2024-05-10T21:06:25.562542+0000","last_clean":"2024-05-10T21:06:25.562542+0000","last_became_active":"2024-05-10T21:04:23.929200+0000","last_became_peered":"2024-05-10T21:04:23.929200+0000","last_unstale":"2024-05-10T21:06:25.562542+0000","last_undegraded":"2024-05-10T21:06:25.562542+0000","last_fullsized":"2024-05-10T21:06:25.562542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:17.574972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558020+0000","last_change":"2024-05-10T21:04:23.927459+0000","last_active":"2024-05-10T21:06:25.558020+0000","last_peered":"2024-05-10T21:06:25.558020+0000","last_clean":"2024-05-10T21:06:25.558020+0000","last_became_active":"2024-05-10T21:04:23.927056+0000","last_became_peered":"2024-05-10T21:04:23.927056+0000","last_unstale":"2024-05-10T21:06:25.558020+0000","last_undegraded":"2024-05-10T21:06:25.558020+0000","last_fullsized":"2024-05-10T21:06:25.558020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:13.173338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.560452+0000","last_change":"2024-05-10T21:04:57.332982+0000","last_active":"2024-05-10T21:06:25.560452+0000","last_peered":"2024-05-10T21:06:25.560452+0000","last_clean":"2024-05-10T21:06:25.560452+0000","last_became_active":"2024-05-10T21:04:57.331919+0000","last_became_peered":"2024-05-10T21:04:57.331919+0000","last_unstale":"2024-05-10T21:06:25.560452+0000","last_undegraded":"2024-05-10T21:06:25.560452+0000","last_fullsized":"2024-05-10T21:06:25.560452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:49:28.437096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567896+0000","last_change":"2024-05-10T21:04:23.922024+0000","last_active":"2024-05-10T21:06:25.567896+0000","last_peered":"2024-05-10T21:06:25.567896+0000","last_clean":"2024-05-10T21:06:25.567896+0000","last_became_active":"2024-05-10T21:04:23.921640+0000","last_became_peered":"2024-05-10T21:04:23.921640+0000","last_unstale":"2024-05-10T21:06:25.567896+0000","last_undegraded":"2024-05-10T21:06:25.567896+0000","last_fullsized":"2024-05-10T21:06:25.567896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:45.066307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:52.442390+0000","last_change":"2024-05-10T21:04:56.896570+0000","last_active":"2024-05-10T21:06:52.442390+0000","last_peered":"2024-05-10T21:06:52.442390+0000","last_clean":"2024-05-10T21:06:52.442390+0000","last_became_active":"2024-05-10T21:04:56.896056+0000","last_became_peered":"2024-05-10T21:04:56.896056+0000","last_unstale":"2024-05-10T21:06:52.442390+0000","last_undegraded":"2024-05-10T21:06:52.442390+0000","last_fullsized":"2024-05-10T21:06:52.442390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:11:23.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566138+0000","last_change":"2024-05-10T21:04:23.923967+0000","last_active":"2024-05-10T21:06:25.566138+0000","last_peered":"2024-05-10T21:06:25.566138+0000","last_clean":"2024-05-10T21:06:25.566138+0000","last_became_active":"2024-05-10T21:04:23.923298+0000","last_became_peered":"2024-05-10T21:04:23.923298+0000","last_unstale":"2024-05-10T21:06:25.566138+0000","last_undegraded":"2024-05-10T21:06:25.566138+0000","last_fullsized":"2024-05-10T21:06:25.566138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:57:12.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.558254+0000","last_change":"2024-05-10T21:04:23.919726+0000","last_active":"2024-05-10T21:06:25.558254+0000","last_peered":"2024-05-10T21:06:25.558254+0000","last_clean":"2024-05-10T21:06:25.558254+0000","last_became_active":"2024-05-10T21:04:23.917742+0000","last_became_peered":"2024-05-10T21:04:23.917742+0000","last_unstale":"2024-05-10T21:06:25.558254+0000","last_undegraded":"2024-05-10T21:06:25.558254+0000","last_fullsized":"2024-05-10T21:06:25.558254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:01.040773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.576351+0000","last_change":"2024-05-10T21:04:23.936690+0000","last_active":"2024-05-10T21:06:25.576351+0000","last_peered":"2024-05-10T21:06:25.576351+0000","last_clean":"2024-05-10T21:06:25.576351+0000","last_became_active":"2024-05-10T21:04:23.936095+0000","last_became_peered":"2024-05-10T21:04:23.936095+0000","last_unstale":"2024-05-10T21:06:25.576351+0000","last_undegraded":"2024-05-10T21:06:25.576351+0000","last_fullsized":"2024-05-10T21:06:25.576351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:26:46.614624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579901+0000","last_change":"2024-05-10T21:04:23.926212+0000","last_active":"2024-05-10T21:06:25.579901+0000","last_peered":"2024-05-10T21:06:25.579901+0000","last_clean":"2024-05-10T21:06:25.579901+0000","last_became_active":"2024-05-10T21:04:23.919262+0000","last_became_peered":"2024-05-10T21:04:23.919262+0000","last_unstale":"2024-05-10T21:06:25.579901+0000","last_undegraded":"2024-05-10T21:06:25.579901+0000","last_fullsized":"2024-05-10T21:06:25.579901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:57.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.579560+0000","last_change":"2024-05-10T21:04:23.920890+0000","last_active":"2024-05-10T21:06:25.579560+0000","last_peered":"2024-05-10T21:06:25.579560+0000","last_clean":"2024-05-10T21:06:25.579560+0000","last_became_active":"2024-05-10T21:04:23.920504+0000","last_became_peered":"2024-05-10T21:04:23.920504+0000","last_unstale":"2024-05-10T21:06:25.579560+0000","last_undegraded":"2024-05-10T21:06:25.579560+0000","last_fullsized":"2024-05-10T21:06:25.579560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:31.862029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581185+0000","last_change":"2024-05-10T21:04:23.926974+0000","last_active":"2024-05-10T21:06:25.581185+0000","last_peered":"2024-05-10T21:06:25.581185+0000","last_clean":"2024-05-10T21:06:25.581185+0000","last_became_active":"2024-05-10T21:04:23.926624+0000","last_became_peered":"2024-05-10T21:04:23.926624+0000","last_unstale":"2024-05-10T21:06:25.581185+0000","last_undegraded":"2024-05-10T21:06:25.581185+0000","last_fullsized":"2024-05-10T21:06:25.581185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:52.786010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.578941+0000","last_change":"2024-05-10T21:04:23.928140+0000","last_active":"2024-05-10T21:06:25.578941+0000","last_peered":"2024-05-10T21:06:25.578941+0000","last_clean":"2024-05-10T21:06:25.578941+0000","last_became_active":"2024-05-10T21:04:23.927175+0000","last_became_peered":"2024-05-10T21:04:23.927175+0000","last_unstale":"2024-05-10T21:06:25.578941+0000","last_undegraded":"2024-05-10T21:06:25.578941+0000","last_fullsized":"2024-05-10T21:06:25.578941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:48:21.101016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.567916+0000","last_change":"2024-05-10T21:04:23.930105+0000","last_active":"2024-05-10T21:06:25.567916+0000","last_peered":"2024-05-10T21:06:25.567916+0000","last_clean":"2024-05-10T21:06:25.567916+0000","last_became_active":"2024-05-10T21:04:23.929740+0000","last_became_peered":"2024-05-10T21:04:23.929740+0000","last_unstale":"2024-05-10T21:06:25.567916+0000","last_undegraded":"2024-05-10T21:06:25.567916+0000","last_fullsized":"2024-05-10T21:06:25.567916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:36:44.604814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.569605+0000","last_change":"2024-05-10T21:04:23.952426+0000","last_active":"2024-05-10T21:06:25.569605+0000","last_peered":"2024-05-10T21:06:25.569605+0000","last_clean":"2024-05-10T21:06:25.569605+0000","last_became_active":"2024-05-10T21:04:23.952015+0000","last_became_peered":"2024-05-10T21:04:23.952015+0000","last_unstale":"2024-05-10T21:06:25.569605+0000","last_undegraded":"2024-05-10T21:06:25.569605+0000","last_fullsized":"2024-05-10T21:06:25.569605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:45:28.718985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:28.939887+0000","last_change":"2024-05-10T21:04:57.183212+0000","last_active":"2024-05-10T21:08:28.939887+0000","last_peered":"2024-05-10T21:08:28.939887+0000","last_clean":"2024-05-10T21:08:28.939887+0000","last_became_active":"2024-05-10T21:04:57.162883+0000","last_became_peered":"2024-05-10T21:04:57.162883+0000","last_unstale":"2024-05-10T21:08:28.939887+0000","last_undegraded":"2024-05-10T21:08:28.939887+0000","last_fullsized":"2024-05-10T21:08:28.939887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:43:34.157481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,1],"acting":[9,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572865+0000","last_change":"2024-05-10T21:04:23.947818+0000","last_active":"2024-05-10T21:06:25.572865+0000","last_peered":"2024-05-10T21:06:25.572865+0000","last_clean":"2024-05-10T21:06:25.572865+0000","last_became_active":"2024-05-10T21:04:23.947441+0000","last_became_peered":"2024-05-10T21:04:23.947441+0000","last_unstale":"2024-05-10T21:06:25.572865+0000","last_undegraded":"2024-05-10T21:06:25.572865+0000","last_fullsized":"2024-05-10T21:06:25.572865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:34.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.886202+0000","last_change":"2024-05-10T21:04:19.868407+0000","last_active":"2024-05-10T21:06:19.886202+0000","last_peered":"2024-05-10T21:06:19.886202+0000","last_clean":"2024-05-10T21:06:19.886202+0000","last_became_active":"2024-05-10T21:04:19.866413+0000","last_became_peered":"2024-05-10T21:04:19.866413+0000","last_unstale":"2024-05-10T21:06:19.886202+0000","last_undegraded":"2024-05-10T21:06:19.886202+0000","last_fullsized":"2024-05-10T21:06:19.886202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:42:41.090509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813423+0000","last_change":"2024-05-10T21:04:23.935808+0000","last_active":"2024-05-10T21:06:28.813423+0000","last_peered":"2024-05-10T21:06:28.813423+0000","last_clean":"2024-05-10T21:06:28.813423+0000","last_became_active":"2024-05-10T21:04:23.934512+0000","last_became_peered":"2024-05-10T21:04:23.934512+0000","last_unstale":"2024-05-10T21:06:28.813423+0000","last_undegraded":"2024-05-10T21:06:28.813423+0000","last_fullsized":"2024-05-10T21:06:28.813423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:53:46.993361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891632+0000","last_change":"2024-05-10T21:04:19.855223+0000","last_active":"2024-05-10T21:04:56.891632+0000","last_peered":"2024-05-10T21:04:56.891632+0000","last_clean":"2024-05-10T21:04:56.891632+0000","last_became_active":"2024-05-10T21:04:19.854782+0000","last_became_peered":"2024-05-10T21:04:19.854782+0000","last_unstale":"2024-05-10T21:04:56.891632+0000","last_undegraded":"2024-05-10T21:04:56.891632+0000","last_fullsized":"2024-05-10T21:04:56.891632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:26:16.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572287+0000","last_change":"2024-05-10T21:04:23.925913+0000","last_active":"2024-05-10T21:06:25.572287+0000","last_peered":"2024-05-10T21:06:25.572287+0000","last_clean":"2024-05-10T21:06:25.572287+0000","last_became_active":"2024-05-10T21:04:23.925565+0000","last_became_peered":"2024-05-10T21:04:23.925565+0000","last_unstale":"2024-05-10T21:06:25.572287+0000","last_undegraded":"2024-05-10T21:06:25.572287+0000","last_fullsized":"2024-05-10T21:06:25.572287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:48.477178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891403+0000","last_change":"2024-05-10T21:04:19.852788+0000","last_active":"2024-05-10T21:04:56.891403+0000","last_peered":"2024-05-10T21:04:56.891403+0000","last_clean":"2024-05-10T21:04:56.891403+0000","last_became_active":"2024-05-10T21:04:19.852396+0000","last_became_peered":"2024-05-10T21:04:19.852396+0000","last_unstale":"2024-05-10T21:04:56.891403+0000","last_undegraded":"2024-05-10T21:04:56.891403+0000","last_fullsized":"2024-05-10T21:04:56.891403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:47:30.206038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.564010+0000","last_change":"2024-05-10T21:04:23.919442+0000","last_active":"2024-05-10T21:06:25.564010+0000","last_peered":"2024-05-10T21:06:25.564010+0000","last_clean":"2024-05-10T21:06:25.564010+0000","last_became_active":"2024-05-10T21:04:23.917459+0000","last_became_peered":"2024-05-10T21:04:23.917459+0000","last_unstale":"2024-05-10T21:06:25.564010+0000","last_undegraded":"2024-05-10T21:06:25.564010+0000","last_fullsized":"2024-05-10T21:06:25.564010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:26:51.743361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":275,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:44.126746+0000","last_change":"2024-05-10T21:04:19.854324+0000","last_active":"2024-05-10T21:08:44.126746+0000","last_peered":"2024-05-10T21:08:44.126746+0000","last_clean":"2024-05-10T21:08:44.126746+0000","last_became_active":"2024-05-10T21:04:19.853216+0000","last_became_peered":"2024-05-10T21:04:19.853216+0000","last_unstale":"2024-05-10T21:08:44.126746+0000","last_undegraded":"2024-05-10T21:08:44.126746+0000","last_fullsized":"2024-05-10T21:08:44.126746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:11.306250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556165+0000","last_change":"2024-05-10T21:04:23.925722+0000","last_active":"2024-05-10T21:06:25.556165+0000","last_peered":"2024-05-10T21:06:25.556165+0000","last_clean":"2024-05-10T21:06:25.556165+0000","last_became_active":"2024-05-10T21:04:23.925362+0000","last_became_peered":"2024-05-10T21:04:23.925362+0000","last_unstale":"2024-05-10T21:06:25.556165+0000","last_undegraded":"2024-05-10T21:06:25.556165+0000","last_fullsized":"2024-05-10T21:06:25.556165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:39:13.941884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.886007+0000","last_change":"2024-05-10T21:04:19.863559+0000","last_active":"2024-05-10T21:04:56.886007+0000","last_peered":"2024-05-10T21:04:56.886007+0000","last_clean":"2024-05-10T21:04:56.886007+0000","last_became_active":"2024-05-10T21:04:19.863233+0000","last_became_peered":"2024-05-10T21:04:19.863233+0000","last_unstale":"2024-05-10T21:04:56.886007+0000","last_undegraded":"2024-05-10T21:04:56.886007+0000","last_fullsized":"2024-05-10T21:04:56.886007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:01:41.431533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.572640+0000","last_change":"2024-05-10T21:04:57.554243+0000","last_active":"2024-05-10T21:06:25.572640+0000","last_peered":"2024-05-10T21:06:25.572640+0000","last_clean":"2024-05-10T21:06:25.572640+0000","last_became_active":"2024-05-10T21:04:57.552477+0000","last_became_peered":"2024-05-10T21:04:57.552477+0000","last_unstale":"2024-05-10T21:06:25.572640+0000","last_undegraded":"2024-05-10T21:06:25.572640+0000","last_fullsized":"2024-05-10T21:06:25.572640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:40:45.114567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.170663+0000","last_change":"2024-05-10T21:04:19.867907+0000","last_active":"2024-05-10T21:04:57.170663+0000","last_peered":"2024-05-10T21:04:57.170663+0000","last_clean":"2024-05-10T21:04:57.170663+0000","last_became_active":"2024-05-10T21:04:19.867475+0000","last_became_peered":"2024-05-10T21:04:19.867475+0000","last_unstale":"2024-05-10T21:04:57.170663+0000","last_undegraded":"2024-05-10T21:04:57.170663+0000","last_fullsized":"2024-05-10T21:04:57.170663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:28:02.511909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574909+0000","last_change":"2024-05-10T21:04:23.949663+0000","last_active":"2024-05-10T21:06:25.574909+0000","last_peered":"2024-05-10T21:06:25.574909+0000","last_clean":"2024-05-10T21:06:25.574909+0000","last_became_active":"2024-05-10T21:04:23.948414+0000","last_became_peered":"2024-05-10T21:04:23.948414+0000","last_unstale":"2024-05-10T21:06:25.574909+0000","last_undegraded":"2024-05-10T21:06:25.574909+0000","last_fullsized":"2024-05-10T21:06:25.574909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:57.312179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330864+0000","last_change":"2024-05-10T21:04:19.850791+0000","last_active":"2024-05-10T21:04:57.330864+0000","last_peered":"2024-05-10T21:04:57.330864+0000","last_clean":"2024-05-10T21:04:57.330864+0000","last_became_active":"2024-05-10T21:04:19.850398+0000","last_became_peered":"2024-05-10T21:04:19.850398+0000","last_unstale":"2024-05-10T21:04:57.330864+0000","last_undegraded":"2024-05-10T21:04:57.330864+0000","last_fullsized":"2024-05-10T21:04:57.330864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:05:46.775894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575638+0000","last_change":"2024-05-10T21:04:23.952321+0000","last_active":"2024-05-10T21:06:25.575638+0000","last_peered":"2024-05-10T21:06:25.575638+0000","last_clean":"2024-05-10T21:06:25.575638+0000","last_became_active":"2024-05-10T21:04:23.951771+0000","last_became_peered":"2024-05-10T21:04:23.951771+0000","last_unstale":"2024-05-10T21:06:25.575638+0000","last_undegraded":"2024-05-10T21:06:25.575638+0000","last_fullsized":"2024-05-10T21:06:25.575638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:41.393826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":272,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:46.128905+0000","last_change":"2024-05-10T21:04:19.849375+0000","last_active":"2024-05-10T21:08:46.128905+0000","last_peered":"2024-05-10T21:08:46.128905+0000","last_clean":"2024-05-10T21:08:46.128905+0000","last_became_active":"2024-05-10T21:04:19.848999+0000","last_became_peered":"2024-05-10T21:04:19.848999+0000","last_unstale":"2024-05-10T21:08:46.128905+0000","last_undegraded":"2024-05-10T21:08:46.128905+0000","last_fullsized":"2024-05-10T21:08:46.128905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:05:46.983158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":241,"num_read_kb":241,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.568418+0000","last_change":"2024-05-10T21:04:23.933312+0000","last_active":"2024-05-10T21:06:25.568418+0000","last_peered":"2024-05-10T21:06:25.568418+0000","last_clean":"2024-05-10T21:06:25.568418+0000","last_became_active":"2024-05-10T21:04:23.932911+0000","last_became_peered":"2024-05-10T21:04:23.932911+0000","last_unstale":"2024-05-10T21:06:25.568418+0000","last_undegraded":"2024-05-10T21:06:25.568418+0000","last_fullsized":"2024-05-10T21:06:25.568418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:50:04.583289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.896268+0000","last_change":"2024-05-10T21:04:19.852137+0000","last_active":"2024-05-10T21:04:56.896268+0000","last_peered":"2024-05-10T21:04:56.896268+0000","last_clean":"2024-05-10T21:04:56.896268+0000","last_became_active":"2024-05-10T21:04:19.851797+0000","last_became_peered":"2024-05-10T21:04:19.851797+0000","last_unstale":"2024-05-10T21:04:56.896268+0000","last_undegraded":"2024-05-10T21:04:56.896268+0000","last_fullsized":"2024-05-10T21:04:56.896268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:56:52.415554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581506+0000","last_change":"2024-05-10T21:04:23.924648+0000","last_active":"2024-05-10T21:06:25.581506+0000","last_peered":"2024-05-10T21:06:25.581506+0000","last_clean":"2024-05-10T21:06:25.581506+0000","last_became_active":"2024-05-10T21:04:23.924032+0000","last_became_peered":"2024-05-10T21:04:23.924032+0000","last_unstale":"2024-05-10T21:06:25.581506+0000","last_undegraded":"2024-05-10T21:06:25.581506+0000","last_fullsized":"2024-05-10T21:06:25.581506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:53:44.699372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330806+0000","last_change":"2024-05-10T21:04:19.858978+0000","last_active":"2024-05-10T21:04:57.330806+0000","last_peered":"2024-05-10T21:04:57.330806+0000","last_clean":"2024-05-10T21:04:57.330806+0000","last_became_active":"2024-05-10T21:04:19.858523+0000","last_became_peered":"2024-05-10T21:04:19.858523+0000","last_unstale":"2024-05-10T21:04:57.330806+0000","last_undegraded":"2024-05-10T21:04:57.330806+0000","last_fullsized":"2024-05-10T21:04:57.330806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:50.304895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.565943+0000","last_change":"2024-05-10T21:04:23.918399+0000","last_active":"2024-05-10T21:06:25.565943+0000","last_peered":"2024-05-10T21:06:25.565943+0000","last_clean":"2024-05-10T21:06:25.565943+0000","last_became_active":"2024-05-10T21:04:23.918007+0000","last_became_peered":"2024-05-10T21:04:23.918007+0000","last_unstale":"2024-05-10T21:06:25.565943+0000","last_undegraded":"2024-05-10T21:06:25.565943+0000","last_fullsized":"2024-05-10T21:06:25.565943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:29:22.228495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'44","reported_seq":568,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:47.283223+0000","last_change":"2024-05-10T21:04:19.853514+0000","last_active":"2024-05-10T21:08:47.283223+0000","last_peered":"2024-05-10T21:08:47.283223+0000","last_clean":"2024-05-10T21:08:47.283223+0000","last_became_active":"2024-05-10T21:04:19.853175+0000","last_became_peered":"2024-05-10T21:04:19.853175+0000","last_unstale":"2024-05-10T21:08:47.283223+0000","last_undegraded":"2024-05-10T21:08:47.283223+0000","last_fullsized":"2024-05-10T21:08:47.283223+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:34:26.526243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73022,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":497,"num_read_kb":318,"num_write":45,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.553044+0000","last_change":"2024-05-10T21:04:23.922999+0000","last_active":"2024-05-10T21:06:25.553044+0000","last_peered":"2024-05-10T21:06:25.553044+0000","last_clean":"2024-05-10T21:06:25.553044+0000","last_became_active":"2024-05-10T21:04:23.922635+0000","last_became_peered":"2024-05-10T21:04:23.922635+0000","last_unstale":"2024-05-10T21:06:25.553044+0000","last_undegraded":"2024-05-10T21:06:25.553044+0000","last_fullsized":"2024-05-10T21:06:25.553044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:59:24.735270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.894679+0000","last_change":"2024-05-10T21:04:19.860267+0000","last_active":"2024-05-10T21:04:56.894679+0000","last_peered":"2024-05-10T21:04:56.894679+0000","last_clean":"2024-05-10T21:04:56.894679+0000","last_became_active":"2024-05-10T21:04:19.859851+0000","last_became_peered":"2024-05-10T21:04:19.859851+0000","last_unstale":"2024-05-10T21:04:56.894679+0000","last_undegraded":"2024-05-10T21:04:56.894679+0000","last_fullsized":"2024-05-10T21:04:56.894679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:08:03.638943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.574567+0000","last_change":"2024-05-10T21:04:23.946553+0000","last_active":"2024-05-10T21:06:25.574567+0000","last_peered":"2024-05-10T21:06:25.574567+0000","last_clean":"2024-05-10T21:06:25.574567+0000","last_became_active":"2024-05-10T21:04:23.945056+0000","last_became_peered":"2024-05-10T21:04:23.945056+0000","last_unstale":"2024-05-10T21:06:25.574567+0000","last_undegraded":"2024-05-10T21:06:25.574567+0000","last_fullsized":"2024-05-10T21:06:25.574567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:39:01.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330487+0000","last_change":"2024-05-10T21:04:19.860595+0000","last_active":"2024-05-10T21:04:57.330487+0000","last_peered":"2024-05-10T21:04:57.330487+0000","last_clean":"2024-05-10T21:04:57.330487+0000","last_became_active":"2024-05-10T21:04:19.860154+0000","last_became_peered":"2024-05-10T21:04:19.860154+0000","last_unstale":"2024-05-10T21:04:57.330487+0000","last_undegraded":"2024-05-10T21:04:57.330487+0000","last_fullsized":"2024-05-10T21:04:57.330487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:09:22.011711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577066+0000","last_change":"2024-05-10T21:04:23.944904+0000","last_active":"2024-05-10T21:06:25.577066+0000","last_peered":"2024-05-10T21:06:25.577066+0000","last_clean":"2024-05-10T21:06:25.577066+0000","last_became_active":"2024-05-10T21:04:23.944537+0000","last_became_peered":"2024-05-10T21:04:23.944537+0000","last_unstale":"2024-05-10T21:06:25.577066+0000","last_undegraded":"2024-05-10T21:06:25.577066+0000","last_fullsized":"2024-05-10T21:06:25.577066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:19:50.056488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":273,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:42.284514+0000","last_change":"2024-05-10T21:04:19.852686+0000","last_active":"2024-05-10T21:08:42.284514+0000","last_peered":"2024-05-10T21:08:42.284514+0000","last_clean":"2024-05-10T21:08:42.284514+0000","last_became_active":"2024-05-10T21:04:19.852341+0000","last_became_peered":"2024-05-10T21:04:19.852341+0000","last_unstale":"2024-05-10T21:08:42.284514+0000","last_undegraded":"2024-05-10T21:08:42.284514+0000","last_fullsized":"2024-05-10T21:08:42.284514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:58.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.549604+0000","last_change":"2024-05-10T21:04:23.919722+0000","last_active":"2024-05-10T21:06:25.549604+0000","last_peered":"2024-05-10T21:06:25.549604+0000","last_clean":"2024-05-10T21:06:25.549604+0000","last_became_active":"2024-05-10T21:04:23.919383+0000","last_became_peered":"2024-05-10T21:04:23.919383+0000","last_unstale":"2024-05-10T21:06:25.549604+0000","last_undegraded":"2024-05-10T21:06:25.549604+0000","last_fullsized":"2024-05-10T21:06:25.549604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:55:20.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.354137+0000","last_change":"2024-05-10T21:04:57.353561+0000","last_active":"2024-05-10T21:04:57.354137+0000","last_peered":"2024-05-10T21:04:57.354137+0000","last_clean":"2024-05-10T21:04:57.354137+0000","last_became_active":"2024-05-10T21:04:57.332682+0000","last_became_peered":"2024-05-10T21:04:57.332682+0000","last_unstale":"2024-05-10T21:04:57.354137+0000","last_undegraded":"2024-05-10T21:04:57.354137+0000","last_fullsized":"2024-05-10T21:04:57.354137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:52:46.589010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.559328+0000","last_change":"2024-05-10T21:04:23.921355+0000","last_active":"2024-05-10T21:06:25.559328+0000","last_peered":"2024-05-10T21:06:25.559328+0000","last_clean":"2024-05-10T21:06:25.559328+0000","last_became_active":"2024-05-10T21:04:23.921018+0000","last_became_peered":"2024-05-10T21:04:23.921018+0000","last_unstale":"2024-05-10T21:06:25.559328+0000","last_undegraded":"2024-05-10T21:06:25.559328+0000","last_fullsized":"2024-05-10T21:06:25.559328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:06.241518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.891741+0000","last_change":"2024-05-10T21:04:19.853182+0000","last_active":"2024-05-10T21:04:56.891741+0000","last_peered":"2024-05-10T21:04:56.891741+0000","last_clean":"2024-05-10T21:04:56.891741+0000","last_became_active":"2024-05-10T21:04:19.852354+0000","last_became_peered":"2024-05-10T21:04:19.852354+0000","last_unstale":"2024-05-10T21:04:56.891741+0000","last_undegraded":"2024-05-10T21:04:56.891741+0000","last_fullsized":"2024-05-10T21:04:56.891741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:02.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556302+0000","last_change":"2024-05-10T21:04:23.920203+0000","last_active":"2024-05-10T21:06:25.556302+0000","last_peered":"2024-05-10T21:06:25.556302+0000","last_clean":"2024-05-10T21:06:25.556302+0000","last_became_active":"2024-05-10T21:04:23.919042+0000","last_became_peered":"2024-05-10T21:04:23.919042+0000","last_unstale":"2024-05-10T21:06:25.556302+0000","last_undegraded":"2024-05-10T21:06:25.556302+0000","last_fullsized":"2024-05-10T21:06:25.556302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:44:37.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:23.917882+0000","last_change":"2024-05-10T21:04:19.853102+0000","last_active":"2024-05-10T21:06:23.917882+0000","last_peered":"2024-05-10T21:06:23.917882+0000","last_clean":"2024-05-10T21:06:23.917882+0000","last_became_active":"2024-05-10T21:04:19.852474+0000","last_became_peered":"2024-05-10T21:04:19.852474+0000","last_unstale":"2024-05-10T21:06:23.917882+0000","last_undegraded":"2024-05-10T21:06:23.917882+0000","last_fullsized":"2024-05-10T21:06:23.917882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:43:36.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:37.454529+0000","last_change":"2024-05-10T21:04:23.947924+0000","last_active":"2024-05-10T21:08:37.454529+0000","last_peered":"2024-05-10T21:08:37.454529+0000","last_clean":"2024-05-10T21:08:37.454529+0000","last_became_active":"2024-05-10T21:04:23.947581+0000","last_became_peered":"2024-05-10T21:04:23.947581+0000","last_unstale":"2024-05-10T21:08:37.454529+0000","last_undegraded":"2024-05-10T21:08:37.454529+0000","last_fullsized":"2024-05-10T21:08:37.454529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:36:45.691563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:24.928701+0000","last_change":"2024-05-10T21:04:19.861844+0000","last_active":"2024-05-10T21:06:24.928701+0000","last_peered":"2024-05-10T21:06:24.928701+0000","last_clean":"2024-05-10T21:06:24.928701+0000","last_became_active":"2024-05-10T21:04:19.861460+0000","last_became_peered":"2024-05-10T21:04:19.861460+0000","last_unstale":"2024-05-10T21:06:24.928701+0000","last_undegraded":"2024-05-10T21:06:24.928701+0000","last_fullsized":"2024-05-10T21:06:24.928701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:41:02.823093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.440071+0000","last_change":"2024-05-10T21:04:23.933315+0000","last_active":"2024-05-10T21:06:32.440071+0000","last_peered":"2024-05-10T21:06:32.440071+0000","last_clean":"2024-05-10T21:06:32.440071+0000","last_became_active":"2024-05-10T21:04:23.932885+0000","last_became_peered":"2024-05-10T21:04:23.932885+0000","last_unstale":"2024-05-10T21:06:32.440071+0000","last_undegraded":"2024-05-10T21:06:32.440071+0000","last_fullsized":"2024-05-10T21:06:32.440071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T00:51:32.226670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.800822+0000","last_change":"2024-05-10T21:04:23.917425+0000","last_active":"2024-05-10T21:06:25.800822+0000","last_peered":"2024-05-10T21:06:25.800822+0000","last_clean":"2024-05-10T21:06:25.800822+0000","last_became_active":"2024-05-10T21:04:23.917052+0000","last_became_peered":"2024-05-10T21:04:23.917052+0000","last_unstale":"2024-05-10T21:06:25.800822+0000","last_undegraded":"2024-05-10T21:06:25.800822+0000","last_fullsized":"2024-05-10T21:06:25.800822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:49:56.721119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":273,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:45.291262+0000","last_change":"2024-05-10T21:04:19.866778+0000","last_active":"2024-05-10T21:08:45.291262+0000","last_peered":"2024-05-10T21:08:45.291262+0000","last_clean":"2024-05-10T21:08:45.291262+0000","last_became_active":"2024-05-10T21:04:19.865928+0000","last_became_peered":"2024-05-10T21:04:19.865928+0000","last_unstale":"2024-05-10T21:08:45.291262+0000","last_undegraded":"2024-05-10T21:08:45.291262+0000","last_fullsized":"2024-05-10T21:08:45.291262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:23.203787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:29.888397+0000","last_change":"2024-05-10T21:04:23.939388+0000","last_active":"2024-05-10T21:06:29.888397+0000","last_peered":"2024-05-10T21:06:29.888397+0000","last_clean":"2024-05-10T21:06:29.888397+0000","last_became_active":"2024-05-10T21:04:23.939005+0000","last_became_peered":"2024-05-10T21:04:23.939005+0000","last_unstale":"2024-05-10T21:06:29.888397+0000","last_undegraded":"2024-05-10T21:06:29.888397+0000","last_fullsized":"2024-05-10T21:06:29.888397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:25:14.559344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.641775+0000","last_change":"2024-05-10T21:04:19.854225+0000","last_active":"2024-05-10T21:06:28.641775+0000","last_peered":"2024-05-10T21:06:28.641775+0000","last_clean":"2024-05-10T21:06:28.641775+0000","last_became_active":"2024-05-10T21:04:19.853888+0000","last_became_peered":"2024-05-10T21:04:19.853888+0000","last_unstale":"2024-05-10T21:06:28.641775+0000","last_undegraded":"2024-05-10T21:06:28.641775+0000","last_fullsized":"2024-05-10T21:06:28.641775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:01:17.829545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:05:19.921438+0000","last_change":"2024-05-10T21:02:43.158535+0000","last_active":"2024-05-10T21:05:19.921438+0000","last_peered":"2024-05-10T21:05:19.921438+0000","last_clean":"2024-05-10T21:05:19.921438+0000","last_became_active":"2024-05-10T21:02:43.152339+0000","last_became_peered":"2024-05-10T21:02:43.152339+0000","last_unstale":"2024-05-10T21:05:19.921438+0000","last_undegraded":"2024-05-10T21:05:19.921438+0000","last_fullsized":"2024-05-10T21:05:19.921438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T20:58:45.926758+0000","last_clean_scrub_stamp":"2024-05-10T20:58:45.926758+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:48:47.352989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756012+0000","last_change":"2024-05-10T21:04:23.941934+0000","last_active":"2024-05-10T21:06:31.756012+0000","last_peered":"2024-05-10T21:06:31.756012+0000","last_clean":"2024-05-10T21:06:31.756012+0000","last_became_active":"2024-05-10T21:04:23.939699+0000","last_became_peered":"2024-05-10T21:04:23.939699+0000","last_unstale":"2024-05-10T21:06:31.756012+0000","last_undegraded":"2024-05-10T21:06:31.756012+0000","last_fullsized":"2024-05-10T21:06:31.756012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:46.353165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:19.928068+0000","last_change":"2024-05-10T21:04:19.850667+0000","last_active":"2024-05-10T21:06:19.928068+0000","last_peered":"2024-05-10T21:06:19.928068+0000","last_clean":"2024-05-10T21:06:19.928068+0000","last_became_active":"2024-05-10T21:04:19.850345+0000","last_became_peered":"2024-05-10T21:04:19.850345+0000","last_unstale":"2024-05-10T21:06:19.928068+0000","last_undegraded":"2024-05-10T21:06:19.928068+0000","last_fullsized":"2024-05-10T21:06:19.928068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:13:40.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:27.832050+0000","last_change":"2024-05-10T21:04:19.865577+0000","last_active":"2024-05-10T21:06:27.832050+0000","last_peered":"2024-05-10T21:06:27.832050+0000","last_clean":"2024-05-10T21:06:27.832050+0000","last_became_active":"2024-05-10T21:04:19.865270+0000","last_became_peered":"2024-05-10T21:04:19.865270+0000","last_unstale":"2024-05-10T21:06:27.832050+0000","last_undegraded":"2024-05-10T21:06:27.832050+0000","last_fullsized":"2024-05-10T21:06:27.832050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:52.537831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.756121+0000","last_change":"2024-05-10T21:04:23.939835+0000","last_active":"2024-05-10T21:06:31.756121+0000","last_peered":"2024-05-10T21:06:31.756121+0000","last_clean":"2024-05-10T21:06:31.756121+0000","last_became_active":"2024-05-10T21:04:23.938650+0000","last_became_peered":"2024-05-10T21:04:23.938650+0000","last_unstale":"2024-05-10T21:06:31.756121+0000","last_undegraded":"2024-05-10T21:06:31.756121+0000","last_fullsized":"2024-05-10T21:06:31.756121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:23:10.905980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:08:16.983580+0000","last_change":"2024-05-10T21:04:19.865477+0000","last_active":"2024-05-10T21:08:16.983580+0000","last_peered":"2024-05-10T21:08:16.983580+0000","last_clean":"2024-05-10T21:08:16.983580+0000","last_became_active":"2024-05-10T21:04:19.864756+0000","last_became_peered":"2024-05-10T21:04:19.864756+0000","last_unstale":"2024-05-10T21:08:16.983580+0000","last_undegraded":"2024-05-10T21:08:16.983580+0000","last_fullsized":"2024-05-10T21:08:16.983580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:41.362227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:34.889491+0000","last_change":"2024-05-10T21:04:23.950257+0000","last_active":"2024-05-10T21:06:34.889491+0000","last_peered":"2024-05-10T21:06:34.889491+0000","last_clean":"2024-05-10T21:06:34.889491+0000","last_became_active":"2024-05-10T21:04:23.948042+0000","last_became_peered":"2024-05-10T21:04:23.948042+0000","last_unstale":"2024-05-10T21:06:34.889491+0000","last_undegraded":"2024-05-10T21:06:34.889491+0000","last_fullsized":"2024-05-10T21:06:34.889491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:19:56.569726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:33.642493+0000","last_change":"2024-05-10T21:04:20.123747+0000","last_active":"2024-05-10T21:06:33.642493+0000","last_peered":"2024-05-10T21:06:33.642493+0000","last_clean":"2024-05-10T21:06:33.642493+0000","last_became_active":"2024-05-10T21:04:20.123418+0000","last_became_peered":"2024-05-10T21:04:20.123418+0000","last_unstale":"2024-05-10T21:06:33.642493+0000","last_undegraded":"2024-05-10T21:06:33.642493+0000","last_fullsized":"2024-05-10T21:06:33.642493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:39:05.682596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:33.814398+0000","last_change":"2024-05-10T21:04:23.942471+0000","last_active":"2024-05-10T21:06:33.814398+0000","last_peered":"2024-05-10T21:06:33.814398+0000","last_clean":"2024-05-10T21:06:33.814398+0000","last_became_active":"2024-05-10T21:04:23.941933+0000","last_became_peered":"2024-05-10T21:04:23.941933+0000","last_unstale":"2024-05-10T21:06:33.814398+0000","last_undegraded":"2024-05-10T21:06:33.814398+0000","last_fullsized":"2024-05-10T21:06:33.814398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:02:52.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:32.154277+0000","last_change":"2024-05-10T21:04:19.858553+0000","last_active":"2024-05-10T21:06:32.154277+0000","last_peered":"2024-05-10T21:06:32.154277+0000","last_clean":"2024-05-10T21:06:32.154277+0000","last_became_active":"2024-05-10T21:04:19.858228+0000","last_became_peered":"2024-05-10T21:04:19.858228+0000","last_unstale":"2024-05-10T21:06:32.154277+0000","last_undegraded":"2024-05-10T21:06:32.154277+0000","last_fullsized":"2024-05-10T21:06:32.154277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:32.851673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.548544+0000","last_change":"2024-05-10T21:04:23.921658+0000","last_active":"2024-05-10T21:06:25.548544+0000","last_peered":"2024-05-10T21:06:25.548544+0000","last_clean":"2024-05-10T21:06:25.548544+0000","last_became_active":"2024-05-10T21:04:23.921258+0000","last_became_peered":"2024-05-10T21:04:23.921258+0000","last_unstale":"2024-05-10T21:06:25.548544+0000","last_undegraded":"2024-05-10T21:06:25.548544+0000","last_fullsized":"2024-05-10T21:06:25.548544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:06.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:29.888329+0000","last_change":"2024-05-10T21:04:19.869668+0000","last_active":"2024-05-10T21:06:29.888329+0000","last_peered":"2024-05-10T21:06:29.888329+0000","last_clean":"2024-05-10T21:06:29.888329+0000","last_became_active":"2024-05-10T21:04:19.868938+0000","last_became_peered":"2024-05-10T21:04:19.868938+0000","last_unstale":"2024-05-10T21:06:29.888329+0000","last_undegraded":"2024-05-10T21:06:29.888329+0000","last_fullsized":"2024-05-10T21:06:29.888329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:42:41.215154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:36.878834+0000","last_change":"2024-05-10T21:04:23.927519+0000","last_active":"2024-05-10T21:06:36.878834+0000","last_peered":"2024-05-10T21:06:36.878834+0000","last_clean":"2024-05-10T21:06:36.878834+0000","last_became_active":"2024-05-10T21:04:23.927134+0000","last_became_peered":"2024-05-10T21:04:23.927134+0000","last_unstale":"2024-05-10T21:06:36.878834+0000","last_undegraded":"2024-05-10T21:06:36.878834+0000","last_fullsized":"2024-05-10T21:06:36.878834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:36:04.817279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814495+0000","last_change":"2024-05-10T21:04:19.861530+0000","last_active":"2024-05-10T21:06:30.814495+0000","last_peered":"2024-05-10T21:06:30.814495+0000","last_clean":"2024-05-10T21:06:30.814495+0000","last_became_active":"2024-05-10T21:04:19.861146+0000","last_became_peered":"2024-05-10T21:04:19.861146+0000","last_unstale":"2024-05-10T21:06:30.814495+0000","last_undegraded":"2024-05-10T21:06:30.814495+0000","last_fullsized":"2024-05-10T21:06:30.814495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:21.754420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.581698+0000","last_change":"2024-05-10T21:04:23.949515+0000","last_active":"2024-05-10T21:06:25.581698+0000","last_peered":"2024-05-10T21:06:25.581698+0000","last_clean":"2024-05-10T21:06:25.581698+0000","last_became_active":"2024-05-10T21:04:23.949176+0000","last_became_peered":"2024-05-10T21:04:23.949176+0000","last_unstale":"2024-05-10T21:06:25.581698+0000","last_undegraded":"2024-05-10T21:06:25.581698+0000","last_fullsized":"2024-05-10T21:06:25.581698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:55.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.341327+0000","last_change":"2024-05-10T21:04:57.340707+0000","last_active":"2024-05-10T21:04:57.341327+0000","last_peered":"2024-05-10T21:04:57.341327+0000","last_clean":"2024-05-10T21:04:57.341327+0000","last_became_active":"2024-05-10T21:04:57.332622+0000","last_became_peered":"2024-05-10T21:04:57.332622+0000","last_unstale":"2024-05-10T21:04:57.341327+0000","last_undegraded":"2024-05-10T21:04:57.341327+0000","last_fullsized":"2024-05-10T21:04:57.341327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:45:00.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:34.930179+0000","last_change":"2024-05-10T21:04:23.937565+0000","last_active":"2024-05-10T21:06:34.930179+0000","last_peered":"2024-05-10T21:06:34.930179+0000","last_clean":"2024-05-10T21:06:34.930179+0000","last_became_active":"2024-05-10T21:04:23.937105+0000","last_became_peered":"2024-05-10T21:04:23.937105+0000","last_unstale":"2024-05-10T21:06:34.930179+0000","last_undegraded":"2024-05-10T21:06:34.930179+0000","last_fullsized":"2024-05-10T21:06:34.930179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:34.781609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:56.895090+0000","last_change":"2024-05-10T21:04:20.124086+0000","last_active":"2024-05-10T21:04:56.895090+0000","last_peered":"2024-05-10T21:04:56.895090+0000","last_clean":"2024-05-10T21:04:56.895090+0000","last_became_active":"2024-05-10T21:04:20.123652+0000","last_became_peered":"2024-05-10T21:04:20.123652+0000","last_unstale":"2024-05-10T21:04:56.895090+0000","last_undegraded":"2024-05-10T21:04:56.895090+0000","last_fullsized":"2024-05-10T21:04:56.895090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:51:18.106024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.566223+0000","last_change":"2024-05-10T21:04:23.922746+0000","last_active":"2024-05-10T21:06:25.566223+0000","last_peered":"2024-05-10T21:06:25.566223+0000","last_clean":"2024-05-10T21:06:25.566223+0000","last_became_active":"2024-05-10T21:04:23.922396+0000","last_became_peered":"2024-05-10T21:04:23.922396+0000","last_unstale":"2024-05-10T21:06:25.566223+0000","last_undegraded":"2024-05-10T21:06:25.566223+0000","last_fullsized":"2024-05-10T21:06:25.566223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:09:53.120789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.330649+0000","last_change":"2024-05-10T21:04:19.864105+0000","last_active":"2024-05-10T21:04:57.330649+0000","last_peered":"2024-05-10T21:04:57.330649+0000","last_clean":"2024-05-10T21:04:57.330649+0000","last_became_active":"2024-05-10T21:04:19.863656+0000","last_became_peered":"2024-05-10T21:04:19.863656+0000","last_unstale":"2024-05-10T21:04:57.330649+0000","last_undegraded":"2024-05-10T21:04:57.330649+0000","last_fullsized":"2024-05-10T21:04:57.330649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:56:03.177000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.575701+0000","last_change":"2024-05-10T21:04:23.941719+0000","last_active":"2024-05-10T21:06:25.575701+0000","last_peered":"2024-05-10T21:06:25.575701+0000","last_clean":"2024-05-10T21:06:25.575701+0000","last_became_active":"2024-05-10T21:04:23.941323+0000","last_became_peered":"2024-05-10T21:04:23.941323+0000","last_unstale":"2024-05-10T21:06:25.575701+0000","last_undegraded":"2024-05-10T21:06:25.575701+0000","last_fullsized":"2024-05-10T21:06:25.575701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:02:57.954752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:04:57.171745+0000","last_change":"2024-05-10T21:04:19.871613+0000","last_active":"2024-05-10T21:04:57.171745+0000","last_peered":"2024-05-10T21:04:57.171745+0000","last_clean":"2024-05-10T21:04:57.171745+0000","last_became_active":"2024-05-10T21:04:19.871206+0000","last_became_peered":"2024-05-10T21:04:19.871206+0000","last_unstale":"2024-05-10T21:04:57.171745+0000","last_undegraded":"2024-05-10T21:04:57.171745+0000","last_fullsized":"2024-05-10T21:04:57.171745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:18.814194+0000","last_clean_scrub_stamp":"2024-05-10T21:04:18.814194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:35:01.239219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.554450+0000","last_change":"2024-05-10T21:04:23.924559+0000","last_active":"2024-05-10T21:06:25.554450+0000","last_peered":"2024-05-10T21:06:25.554450+0000","last_clean":"2024-05-10T21:06:25.554450+0000","last_became_active":"2024-05-10T21:04:23.924010+0000","last_became_peered":"2024-05-10T21:04:23.924010+0000","last_unstale":"2024-05-10T21:06:25.554450+0000","last_undegraded":"2024-05-10T21:06:25.554450+0000","last_fullsized":"2024-05-10T21:06:25.554450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:06.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:31.878305+0000","last_change":"2024-05-10T21:04:23.918671+0000","last_active":"2024-05-10T21:06:31.878305+0000","last_peered":"2024-05-10T21:06:31.878305+0000","last_clean":"2024-05-10T21:06:31.878305+0000","last_became_active":"2024-05-10T21:04:23.917855+0000","last_became_peered":"2024-05-10T21:04:23.917855+0000","last_unstale":"2024-05-10T21:06:31.878305+0000","last_undegraded":"2024-05-10T21:06:31.878305+0000","last_fullsized":"2024-05-10T21:06:31.878305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:43.366019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814417+0000","last_change":"2024-05-10T21:04:23.921772+0000","last_active":"2024-05-10T21:06:30.814417+0000","last_peered":"2024-05-10T21:06:30.814417+0000","last_clean":"2024-05-10T21:06:30.814417+0000","last_became_active":"2024-05-10T21:04:23.921407+0000","last_became_peered":"2024-05-10T21:04:23.921407+0000","last_unstale":"2024-05-10T21:06:30.814417+0000","last_undegraded":"2024-05-10T21:06:30.814417+0000","last_fullsized":"2024-05-10T21:06:30.814417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:13:59.937086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:30.814456+0000","last_change":"2024-05-10T21:04:23.946668+0000","last_active":"2024-05-10T21:06:30.814456+0000","last_peered":"2024-05-10T21:06:30.814456+0000","last_clean":"2024-05-10T21:06:30.814456+0000","last_became_active":"2024-05-10T21:04:23.946249+0000","last_became_peered":"2024-05-10T21:04:23.946249+0000","last_unstale":"2024-05-10T21:06:30.814456+0000","last_undegraded":"2024-05-10T21:06:30.814456+0000","last_fullsized":"2024-05-10T21:06:30.814456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:11:18.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:28.813238+0000","last_change":"2024-05-10T21:04:23.941413+0000","last_active":"2024-05-10T21:06:28.813238+0000","last_peered":"2024-05-10T21:06:28.813238+0000","last_clean":"2024-05-10T21:06:28.813238+0000","last_became_active":"2024-05-10T21:04:23.940943+0000","last_became_peered":"2024-05-10T21:04:23.940943+0000","last_unstale":"2024-05-10T21:06:28.813238+0000","last_undegraded":"2024-05-10T21:06:28.813238+0000","last_fullsized":"2024-05-10T21:06:28.813238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:09:04.893758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:36.757007+0000","last_change":"2024-05-10T21:04:23.935134+0000","last_active":"2024-05-10T21:06:36.757007+0000","last_peered":"2024-05-10T21:06:36.757007+0000","last_clean":"2024-05-10T21:06:36.757007+0000","last_became_active":"2024-05-10T21:04:23.934730+0000","last_became_peered":"2024-05-10T21:04:23.934730+0000","last_unstale":"2024-05-10T21:06:36.757007+0000","last_undegraded":"2024-05-10T21:06:36.757007+0000","last_fullsized":"2024-05-10T21:06:36.757007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:58:36.493379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:36.878878+0000","last_change":"2024-05-10T21:04:23.920234+0000","last_active":"2024-05-10T21:06:36.878878+0000","last_peered":"2024-05-10T21:06:36.878878+0000","last_clean":"2024-05-10T21:06:36.878878+0000","last_became_active":"2024-05-10T21:04:23.919838+0000","last_became_peered":"2024-05-10T21:04:23.919838+0000","last_unstale":"2024-05-10T21:06:36.878878+0000","last_undegraded":"2024-05-10T21:06:36.878878+0000","last_fullsized":"2024-05-10T21:06:36.878878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:56:20.140987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:35.815057+0000","last_change":"2024-05-10T21:04:23.921732+0000","last_active":"2024-05-10T21:06:35.815057+0000","last_peered":"2024-05-10T21:06:35.815057+0000","last_clean":"2024-05-10T21:06:35.815057+0000","last_became_active":"2024-05-10T21:04:23.921305+0000","last_became_peered":"2024-05-10T21:04:23.921305+0000","last_unstale":"2024-05-10T21:06:35.815057+0000","last_undegraded":"2024-05-10T21:06:35.815057+0000","last_fullsized":"2024-05-10T21:06:35.815057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:08.677888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:34.889420+0000","last_change":"2024-05-10T21:04:23.950021+0000","last_active":"2024-05-10T21:06:34.889420+0000","last_peered":"2024-05-10T21:06:34.889420+0000","last_clean":"2024-05-10T21:06:34.889420+0000","last_became_active":"2024-05-10T21:04:23.949518+0000","last_became_peered":"2024-05-10T21:04:23.949518+0000","last_unstale":"2024-05-10T21:06:34.889420+0000","last_undegraded":"2024-05-10T21:06:34.889420+0000","last_fullsized":"2024-05-10T21:06:34.889420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:34:37.277314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:35.801949+0000","last_change":"2024-05-10T21:04:23.939670+0000","last_active":"2024-05-10T21:06:35.801949+0000","last_peered":"2024-05-10T21:06:35.801949+0000","last_clean":"2024-05-10T21:06:35.801949+0000","last_became_active":"2024-05-10T21:04:23.939271+0000","last_became_peered":"2024-05-10T21:04:23.939271+0000","last_unstale":"2024-05-10T21:06:35.801949+0000","last_undegraded":"2024-05-10T21:06:35.801949+0000","last_fullsized":"2024-05-10T21:06:35.801949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:24:15.573263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.571584+0000","last_change":"2024-05-10T21:04:23.947341+0000","last_active":"2024-05-10T21:06:25.571584+0000","last_peered":"2024-05-10T21:06:25.571584+0000","last_clean":"2024-05-10T21:06:25.571584+0000","last_became_active":"2024-05-10T21:04:23.946948+0000","last_became_peered":"2024-05-10T21:04:23.946948+0000","last_unstale":"2024-05-10T21:06:25.571584+0000","last_undegraded":"2024-05-10T21:06:25.571584+0000","last_fullsized":"2024-05-10T21:06:25.571584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:25:08.065626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.577288+0000","last_change":"2024-05-10T21:04:23.944677+0000","last_active":"2024-05-10T21:06:25.577288+0000","last_peered":"2024-05-10T21:06:25.577288+0000","last_clean":"2024-05-10T21:06:25.577288+0000","last_became_active":"2024-05-10T21:04:23.944255+0000","last_became_peered":"2024-05-10T21:04:23.944255+0000","last_unstale":"2024-05-10T21:06:25.577288+0000","last_undegraded":"2024-05-10T21:06:25.577288+0000","last_fullsized":"2024-05-10T21:06:25.577288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:58:18.002076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.580966+0000","last_change":"2024-05-10T21:04:23.939945+0000","last_active":"2024-05-10T21:06:25.580966+0000","last_peered":"2024-05-10T21:06:25.580966+0000","last_clean":"2024-05-10T21:06:25.580966+0000","last_became_active":"2024-05-10T21:04:23.939531+0000","last_became_peered":"2024-05-10T21:04:23.939531+0000","last_unstale":"2024-05-10T21:06:25.580966+0000","last_undegraded":"2024-05-10T21:06:25.580966+0000","last_fullsized":"2024-05-10T21:06:25.580966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:44:44.333339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-10T21:06:25.556196+0000","last_change":"2024-05-10T21:04:23.921835+0000","last_active":"2024-05-10T21:06:25.556196+0000","last_peered":"2024-05-10T21:06:25.556196+0000","last_clean":"2024-05-10T21:06:25.556196+0000","last_became_active":"2024-05-10T21:04:23.921475+0000","last_became_peered":"2024-05-10T21:04:23.921475+0000","last_unstale":"2024-05-10T21:06:25.556196+0000","last_undegraded":"2024-05-10T21:06:25.556196+0000","last_fullsized":"2024-05-10T21:06:25.556196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T21:04:22.875504+0000","last_clean_scrub_stamp":"2024-05-10T21:04:22.875504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:39.874053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77474,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1474,"num_read_kb":819,"num_write":188,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":226678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645385,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":437607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841612,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1099180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037841,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":437261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234072,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":436599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430303,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1495040,"data_stored":1028980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":436650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":437921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":508133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":436961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822761,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1028714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T21:08:49.752 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:08:50.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:49 smithi092 ceph-mon[34415]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:50.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:49 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:08:50.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:49 smithi142 ceph-mon[34846]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:50.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:49 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:08:50.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:49 smithi050 ceph-mon[29546]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T21:08:50.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:49 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:08:50.278 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:51.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:50 smithi092 ceph-mon[34415]: pgmap v459: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:08:51.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:50 smithi142 ceph-mon[34846]: pgmap v459: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:08:51.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:50 smithi050 ceph-mon[29546]: pgmap v459: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:08:52.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 -- 172.21.15.50:0/3514021085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106570 msgr2=0x7fa7c0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:52.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3514021085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106570 0x7fa7c0106980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009990 tx=0x7fa7a8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 -- 172.21.15.50:0/3514021085 shutdown_connections 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3514021085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106ec0 0x7fa7c0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3514021085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3514021085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106570 0x7fa7c0106980 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 -- 172.21.15.50:0/3514021085 >> 172.21.15.50:0/3514021085 conn(0x7fa7c0102090 msgr2=0x7fa7c01044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 -- 172.21.15.50:0/3514021085 shutdown_connections 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.628+0000 7fa7c5200700 1 -- 172.21.15.50:0/3514021085 wait complete. 2024-05-10T21:08:52.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.629+0000 7fa7c5200700 1 Processor -- start 2024-05-10T21:08:52.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 -- start start 2024-05-10T21:08:52.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:52.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 0x7fa7c019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c01166e0 con 0x7fa7c010c5b0 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0116560 con 0x7fa7c0106ec0 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.630+0000 7fa7c5200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0116860 con 0x7fa7c0106570 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7b7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 0x7fa7c019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:52.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54488/0 (socket says 172.21.15.50:54488) 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 -- 172.21.15.50:0/3908219886 learned_addr learned my addr 172.21.15.50:0/3908219886 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7be800700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 -- 172.21.15.50:0/3908219886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 msgr2=0x7fa7c019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 0x7fa7c019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 -- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 msgr2=0x7fa7c01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7bde00700 1 -- 172.21.15.50:0/3908219886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c01a1f20 con 0x7fa7c0106570 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7be800700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:08:52.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.631+0000 7fa7b7e00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 0x7fa7c019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:52.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7bde00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e970 tx=0x7fa7b000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000cbd0 con 0x7fa7c0106570 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b000cd70 con 0x7fa7c0106570 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c01a2210 con 0x7fa7c0106570 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0005680 con 0x7fa7c0106570 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.632+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01a9b30 con 0x7fa7c0106570 2024-05-10T21:08:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.633+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c00512c0 con 0x7fa7c0106570 2024-05-10T21:08:52.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.636+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa7b0047960 con 0x7fa7c0106570 2024-05-10T21:08:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.638+0000 7fa7b6a00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 0x7fa7a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.638+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa7b00e2460 con 0x7fa7c0106570 2024-05-10T21:08:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.638+0000 7fa7b7e00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 0x7fa7a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.639+0000 7fa7b7e00700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 0x7fa7a40b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006010 tx=0x7fa7a800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.640+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7b00aa150 con 0x7fa7c0106570 2024-05-10T21:08:52.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:52 smithi050 ceph-mon[29546]: pgmap v460: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:08:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.927+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7c019e3b0 con 0x7fa7c0106570 2024-05-10T21:08:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.928+0000 7fa7b6a00700 1 -- 172.21.15.50:0/3908219886 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fa7b00a9a50 con 0x7fa7c0106570 2024-05-10T21:08:52.929 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:52.929 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.931+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 msgr2=0x7fa7a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.931+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 0x7fa7a40b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006010 tx=0x7fa7a800b260 comp rx=0 tx=0).stop 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.931+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 msgr2=0x7fa7c019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.931+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e970 tx=0x7fa7b000ec80 comp rx=0 tx=0).stop 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 shutdown_connections 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa7a40b2880 0x7fa7a40b4d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7c010c5b0 0x7fa7c01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c0106570 0x7fa7c019d050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 --2- 172.21.15.50:0/3908219886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7c0106ec0 0x7fa7c019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 >> 172.21.15.50:0/3908219886 conn(0x7fa7c0102090 msgr2=0x7fa7c0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 shutdown_connections 2024-05-10T21:08:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:52.932+0000 7fa7c5200700 1 -- 172.21.15.50:0/3908219886 wait complete. 2024-05-10T21:08:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:08:53.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:52 smithi092 ceph-mon[34415]: pgmap v460: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:08:53.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:52 smithi142 ceph-mon[34846]: pgmap v460: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:08:53.763 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:08:54.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:53 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3908219886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:54.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:53 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3908219886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:54.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:53 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3908219886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:54.234 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:55.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:54 smithi092 ceph-mon[34415]: pgmap v461: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:08:55.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:54 smithi142 ceph-mon[34846]: pgmap v461: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:08:55.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:54 smithi050 ceph-mon[29546]: pgmap v461: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 -- 172.21.15.50:0/2641881367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03810e8e0 msgr2=0x7fb038110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2641881367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03810e8e0 0x7fb038110dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb030009f90 tx=0x7fb03000b330 comp rx=0 tx=0).stop 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 -- 172.21.15.50:0/2641881367 shutdown_connections 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2641881367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03810e8e0 0x7fb038110dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2641881367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105500 0x7fb038105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2641881367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb038105e50 0x7fb03810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 -- 172.21.15.50:0/2641881367 >> 172.21.15.50:0/2641881367 conn(0x7fb0380fedc0 msgr2=0x7fb0381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 -- 172.21.15.50:0/2641881367 shutdown_connections 2024-05-10T21:08:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.688+0000 7fb03ec00700 1 -- 172.21.15.50:0/2641881367 wait complete. 2024-05-10T21:08:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.689+0000 7fb03ec00700 1 Processor -- start 2024-05-10T21:08:56.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.689+0000 7fb03ec00700 1 -- start start 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 0x7fb03819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 0x7fb03819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 0x7fb0381a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114490 con 0x7fb038105e50 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114310 con 0x7fb03810e8e0 2024-05-10T21:08:56.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb03ec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114610 con 0x7fb038105500 2024-05-10T21:08:56.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb02fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 0x7fb03819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb037e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 0x7fb03819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.690+0000 7fb02fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 0x7fb03819d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58112/0 (socket says 172.21.15.50:58112) 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb02fe00700 1 -- 172.21.15.50:0/2007071110 learned_addr learned my addr 172.21.15.50:0/2007071110 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb03cc00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 0x7fb0381a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 -- 172.21.15.50:0/2007071110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 msgr2=0x7fb0381a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 0x7fb0381a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 -- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 msgr2=0x7fb03819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 0x7fb03819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 -- 172.21.15.50:0/2007071110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0381a1e50 con 0x7fb038105500 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb03cc00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 0x7fb0381a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.691+0000 7fb037e00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 0x7fb03819cf80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9c0 tx=0x7fb02800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.692+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02800c9c0 con 0x7fb038105500 2024-05-10T21:08:56.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.692+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02800cb60 con 0x7fb038105500 2024-05-10T21:08:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.692+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0381a2140 con 0x7fb038105500 2024-05-10T21:08:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.692+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028018420 con 0x7fb038105500 2024-05-10T21:08:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.692+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a9a60 con 0x7fb038105500 2024-05-10T21:08:56.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.693+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0380512c0 con 0x7fb038105500 2024-05-10T21:08:56.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.696+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb028010430 con 0x7fb038105500 2024-05-10T21:08:56.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.697+0000 7fb035600700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 0x7fb01c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:08:56.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.698+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb0280e2810 con 0x7fb038105500 2024-05-10T21:08:56.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.698+0000 7fb02fe00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 0x7fb01c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:08:56.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.698+0000 7fb02fe00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 0x7fb01c0b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0381106f0 tx=0x7fb020009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:08:56.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.699+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0280aa500 con 0x7fb038105500 2024-05-10T21:08:56.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.992+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0381a9c30 con 0x7fb038105500 2024-05-10T21:08:56.993 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:56 smithi050 ceph-mon[29546]: pgmap v462: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:08:56.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.994+0000 7fb035600700 1 -- 172.21.15.50:0/2007071110 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fb0280a9e00 con 0x7fb038105500 2024-05-10T21:08:56.993 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:08:56.994 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:08:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 msgr2=0x7fb01c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 0x7fb01c0b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0381106f0 tx=0x7fb020009260 comp rx=0 tx=0).stop 2024-05-10T21:08:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 msgr2=0x7fb03819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:08:56.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 0x7fb03819cf80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9c0 tx=0x7fb02800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 shutdown_connections 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb01c0b2880 0x7fb01c0b4d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03810e8e0 0x7fb0381a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb038105500 0x7fb03819cf80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 --2- 172.21.15.50:0/2007071110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb038105e50 0x7fb03819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 >> 172.21.15.50:0/2007071110 conn(0x7fb0380fedc0 msgr2=0x7fb038103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.997+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 shutdown_connections 2024-05-10T21:08:56.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:08:56.998+0000 7fb03ec00700 1 -- 172.21.15.50:0/2007071110 wait complete. 2024-05-10T21:08:56.998 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:08:57.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:56 smithi092 ceph-mon[34415]: pgmap v462: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:08:57.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:56 smithi142 ceph-mon[34846]: pgmap v462: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:08:57.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:08:58.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:57 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2007071110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:58.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:57 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2007071110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:58.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:57 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2007071110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:08:58.267 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:08:59.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:08:58 smithi092 ceph-mon[34415]: pgmap v463: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:08:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:08:58 smithi142 ceph-mon[34846]: pgmap v463: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:08:59.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:08:58 smithi050 ceph-mon[29546]: pgmap v463: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 -- 172.21.15.50:0/2461576951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c106b90 msgr2=0x7efd9c109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 --2- 172.21.15.50:0/2461576951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c106b90 0x7efd9c109070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efd84009900 tx=0x7efd8402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 -- 172.21.15.50:0/2461576951 shutdown_connections 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 --2- 172.21.15.50:0/2461576951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c109700 0x7efd9c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 --2- 172.21.15.50:0/2461576951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c106b90 0x7efd9c109070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 --2- 172.21.15.50:0/2461576951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 -- 172.21.15.50:0/2461576951 >> 172.21.15.50:0/2461576951 conn(0x7efd9c0fdb60 msgr2=0x7efd9c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 -- 172.21.15.50:0/2461576951 shutdown_connections 2024-05-10T21:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.512+0000 7efda1600640 1 -- 172.21.15.50:0/2461576951 wait complete. 2024-05-10T21:09:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 Processor -- start 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 -- start start 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 0x7efd9c1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c109700 0x7efd9c1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c10f570 con 0x7efd9c109700 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c10f3f0 con 0x7efd9c1041a0 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.514+0000 7efda1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c10f6f0 con 0x7efd9c106b90 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 0x7efd9c1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 0x7efd9c1020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42678/0 (socket says 172.21.15.50:42678) 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44272/0 (socket says 172.21.15.50:44272) 2024-05-10T21:09:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd93e00640 1 -- 172.21.15.50:0/4035090130 learned_addr learned my addr 172.21.15.50:0/4035090130 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efda0c00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c109700 0x7efd9c1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 -- 172.21.15.50:0/4035090130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 msgr2=0x7efd9c1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 0x7efd9c1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 -- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c109700 msgr2=0x7efd9c1025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c109700 0x7efd9c1025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.515+0000 7efd9be00640 1 -- 172.21.15.50:0/4035090130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c102d20 con 0x7efd9c1041a0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efd9be00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9c0 tx=0x7efd8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efd99600640 1 -- 172.21.15.50:0/4035090130 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00cd80 con 0x7efd9c1041a0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c1a4200 con 0x7efd9c1041a0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efd99600640 1 -- 172.21.15.50:0/4035090130 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd8c0108e0 con 0x7efd9c1041a0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efd99600640 1 -- 172.21.15.50:0/4035090130 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0187f0 con 0x7efd9c1041a0 2024-05-10T21:09:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.516+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c1a46f0 con 0x7efd9c1041a0 2024-05-10T21:09:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.519+0000 7efda9c0d740 1 Processor -- start 2024-05-10T21:09:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.519+0000 7efda9c0d740 1 -- start start 2024-05-10T21:09:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647c992f0 0x558648064880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x55864806b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558647fc8550 con 0x558647ca14d0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558647f6dbc0 con 0x558647cc21b0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda9c0d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558647fc4510 con 0x558647c992f0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda0c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda0c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44284/0 (socket says 172.21.15.50:44284) 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efda0c00640 1 -- 172.21.15.50:0/4007288201 learned_addr learned my addr 172.21.15.50:0/4007288201 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efd93e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x55864806b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.520+0000 7efd99600640 1 -- 172.21.15.50:0/4035090130 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd8c01f020 con 0x7efd9c1041a0 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 -- 172.21.15.50:0/4007288201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647c992f0 msgr2=0x558648064880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647c992f0 0x558648064880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 -- 172.21.15.50:0/4007288201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 msgr2=0x55864806b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x55864806b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 -- 172.21.15.50:0/4007288201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558647cda210 con 0x558647cc21b0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efda0c00640 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd94009fd0 tx=0x7efd940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a990885bfc6a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.521+0000 7efd91200640 1 -- 172.21.15.50:0/4007288201 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94004d20 con 0x558647cc21b0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.522+0000 7efd91200640 1 -- 172.21.15.50:0/4007288201 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd94004050 con 0x558647cc21b0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.522+0000 7efd91200640 1 -- 172.21.15.50:0/4007288201 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94005260 con 0x558647cc21b0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.522+0000 7efd99600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 0x7efd800b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.522+0000 7efda9c0d740 1 -- 172.21.15.50:0/4007288201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 msgr2=0x558648064300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.522+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd94009fd0 tx=0x7efd940118d0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efd99600640 1 -- 172.21.15.50:0/4035090130 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efd8c0e2120 con 0x7efd9c1041a0 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 -- 172.21.15.50:0/4007288201 shutdown_connections 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647c992f0 0x558648064880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efd9be00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 0x7efd800b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x55864806b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4007288201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648064300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 -- 172.21.15.50:0/4007288201 >> 172.21.15.50:0/4007288201 conn(0x558647cb6300 msgr2=0x55864805fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 -- 172.21.15.50:0/4007288201 shutdown_connections 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efda9c0d740 1 -- 172.21.15.50:0/4007288201 wait complete. 2024-05-10T21:09:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.523+0000 7efd9be00640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 0x7efd800b4d70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0a9410 tx=0x7efd8c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.524+0000 7efda9c0d740 1 -- start start 2024-05-10T21:09:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.524+0000 7efda9c0d740 1 Processor -- start 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 0x55864806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x55864806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648075230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55864806c1d0 con 0x558647c992f0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558647f81660 con 0x558647cc21b0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55864806c350 con 0x558647ca14d0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda9c0d740 10 client.0 _ll_register_callbacks cb 0x558647f4b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 0x55864806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 0x55864806dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58136/0 (socket says 172.21.15.50:58136) 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efda0c00640 1 -- 172.21.15.50:0/2004249770 learned_addr learned my addr 172.21.15.50:0/2004249770 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.525+0000 7efd93e00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x55864806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efda0c00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648075230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 -- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 msgr2=0x558648075230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648075230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 -- 172.21.15.50:0/2004249770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 msgr2=0x55864806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 0x55864806dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 -- 172.21.15.50:0/2004249770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558647da6620 con 0x558647ca14d0 2024-05-10T21:09:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efda0c00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648075230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:09:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd93e00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x55864806e2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd9c1033b0 tx=0x7efd84031e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f14f58275f098c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd89600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.526+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84035040 con 0x558647ca14d0 2024-05-10T21:09:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.527+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94004050 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.527+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efd84004040 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.527+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd84004240 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.527+0000 7efda9c0d740 10 client.25198 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.527+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558648078220 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd94004050 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efd84044020 con 0x558647ca14d0 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 10 client.25198 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 10 client.25198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 20 client.25198 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c80d80 0x55864807b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 4 client.25198 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.528+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558647fd6160 con 0x558647c80d80 2024-05-10T21:09:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.529+0000 7efda0c00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c80d80 0x55864807b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.529+0000 7efda0c00640 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c80d80 0x55864807b860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b12803f2afaa2d0 server_cookie=30c1696710d8e215 in_seq=0 out_seq=0 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.529+0000 7efd89600640 10 client.25198 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efd89600640 1 -- 172.21.15.50:0/2004249770 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558647fd6160 con 0x558647c80d80 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efd89600640 10 client.25198 handle_command_reply: tid=0 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 client.25198 shutdown 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 msgr2=0x55864806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x55864806e2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd9c1033b0 tx=0x7efd84031e10 comp rx=0 tx=0).stop 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 shutdown_connections 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c80d80 0x55864807b860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x55864806e2d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647cc21b0 0x558648075230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 --2- 172.21.15.50:0/2004249770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647c992f0 0x55864806dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efd89600640 0 client.25198 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.530+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 >> 172.21.15.50:0/2004249770 conn(0x558647cb6300 msgr2=0x55864805ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.531+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 shutdown_connections 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.531+0000 7efda9c0d740 1 -- 172.21.15.50:0/2004249770 wait complete. 2024-05-10T21:09:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.531+0000 7efda9c0d740 20 client.25198 trim_cache size 0 max 16384 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 -- 172.21.15.50:0/3175610494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2408109870 msgr2=0x7f24081111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 --2- 172.21.15.50:0/3175610494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2408109870 0x7f24081111d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2400009fc0 tx=0x7f240000b330 comp rx=0 tx=0).stop 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 -- 172.21.15.50:0/3175610494 shutdown_connections 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 --2- 172.21.15.50:0/3175610494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24081045f0 0x7f2408104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 --2- 172.21.15.50:0/3175610494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2408109870 0x7f24081111d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 --2- 172.21.15.50:0/3175610494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 0x7f2408109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 -- 172.21.15.50:0/3175610494 >> 172.21.15.50:0/3175610494 conn(0x7f2408100040 msgr2=0x7f2408102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 -- 172.21.15.50:0/3175610494 shutdown_connections 2024-05-10T21:09:00.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.538+0000 7f240f800700 1 -- 172.21.15.50:0/3175610494 wait complete. 2024-05-10T21:09:00.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 Processor -- start 2024-05-10T21:09:00.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 -- start start 2024-05-10T21:09:00.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 0x7f240819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 0x7f240819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24081144f0 con 0x7f2408108f20 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2408114370 con 0x7f2408109870 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.540+0000 7f240f800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2408114670 con 0x7f24081045f0 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 0x7f240819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44310/0 (socket says 172.21.15.50:44310) 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 0x7f240819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42718/0 (socket says 172.21.15.50:42718) 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 -- 172.21.15.50:0/3092282080 learned_addr learned my addr 172.21.15.50:0/3092282080 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f2407e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 0x7f240819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 -- 172.21.15.50:0/3092282080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 msgr2=0x7f240819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 0x7f240819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 -- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 msgr2=0x7f240819d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 0x7f240819d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.541+0000 7f240d800700 1 -- 172.21.15.50:0/3092282080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24081a2030 con 0x7f2408109870 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f240d800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2400009790 tx=0x7f240000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400021070 con 0x7f2408109870 2024-05-10T21:09:00.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240000ce60 con 0x7f2408109870 2024-05-10T21:09:00.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24081a22c0 con 0x7f2408109870 2024-05-10T21:09:00.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400005040 con 0x7f2408109870 2024-05-10T21:09:00.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.542+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24081a9c40 con 0x7f2408109870 2024-05-10T21:09:00.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.544+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24080512c0 con 0x7f2408109870 2024-05-10T21:09:00.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.546+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2400024070 con 0x7f2408109870 2024-05-10T21:09:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.547+0000 7f2405600700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 0x7f23f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.548+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f240000f020 con 0x7f2408109870 2024-05-10T21:09:00.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.548+0000 7f240ce00700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 0x7f23f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.549+0000 7f240ce00700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 0x7f23f00b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009c60 tx=0x7f23fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.550+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f240002b020 con 0x7f2408109870 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 Processor -- start 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 -- start start 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x55864806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647cc21b0 0x558648081060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1afb00 con 0x558647cc21b0 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ad3d0 con 0x558647c992f0 2024-05-10T21:09:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.552+0000 7efda9c0d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ab220 con 0x558647ca14d0 2024-05-10T21:09:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efda0c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x55864806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efda0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647cc21b0 0x558648081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42726/0 (socket says 172.21.15.50:42726) 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 -- 172.21.15.50:0/1104039217 learned_addr learned my addr 172.21.15.50:0/1104039217 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 -- 172.21.15.50:0/1104039217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 msgr2=0x55864806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x55864806cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 -- 172.21.15.50:0/1104039217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647cc21b0 msgr2=0x558648081060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647cc21b0 0x558648081060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 -- 172.21.15.50:0/1104039217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864805b4a0 con 0x558647ca14d0 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efda0c00640 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x55864806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.553+0000 7efd93e00640 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd84046510 tx=0x7efd8402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c9a1de481d9e6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.554+0000 7efd8be00640 1 -- 172.21.15.50:0/1104039217 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84000e60 con 0x558647ca14d0 2024-05-10T21:09:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.554+0000 7efd8be00640 1 -- 172.21.15.50:0/1104039217 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd840054e0 con 0x558647ca14d0 2024-05-10T21:09:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.554+0000 7efd8be00640 1 -- 172.21.15.50:0/1104039217 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84035040 con 0x558647ca14d0 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- 172.21.15.50:0/1104039217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 msgr2=0x5586480631a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd84046510 tx=0x7efd8402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- 172.21.15.50:0/1104039217 shutdown_connections 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647ca14d0 0x5586480631a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647cc21b0 0x558648081060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 --2- 172.21.15.50:0/1104039217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x55864806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- 172.21.15.50:0/1104039217 >> 172.21.15.50:0/1104039217 conn(0x558647cb6300 msgr2=0x558648062a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- 172.21.15.50:0/1104039217 shutdown_connections 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- 172.21.15.50:0/1104039217 wait complete. 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 -- start start 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.555+0000 7efda9c0d740 1 Processor -- start 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x558648073570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647cc21b0 0x558648074430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ab890 con 0x558647ca14d0 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1acd00 con 0x558647c992f0 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1abf60 con 0x558647cc21b0 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda9c0d740 10 client.0 _ll_register_callbacks cb 0x7efd84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda0c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda0c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44318/0 (socket says 172.21.15.50:44318) 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda0c00640 1 -- 172.21.15.50:0/4221048035 learned_addr learned my addr 172.21.15.50:0/4221048035 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efd93e00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x558648073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.556+0000 7efda0c00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647cc21b0 0x558648074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 -- 172.21.15.50:0/4221048035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647cc21b0 msgr2=0x558648074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647cc21b0 0x558648074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 -- 172.21.15.50:0/4221048035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 msgr2=0x558648073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x558648073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 -- 172.21.15.50:0/4221048035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586480749f0 con 0x558647c992f0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda0c00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd94003fd0 tx=0x7efd940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df827affe0ac03c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efd89600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94004490 con 0x558647c992f0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.557+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd840054e0 con 0x558647c992f0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efd9400f4e0 con 0x558647c992f0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd94004630 con 0x558647c992f0 2024-05-10T21:09:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 10 client.25214 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558648075cd0 con 0x558647c992f0 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd840054e0 con 0x558647c992f0 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efd94028710 con 0x558647c992f0 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 10 client.25214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 20 client.25214 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.558+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c561c0 0x5586480e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.559+0000 7efda9c0d740 4 client.25214 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:09:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.559+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558647fcffd0 con 0x558647c561c0 2024-05-10T21:09:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.559+0000 7efd93e00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c561c0 0x5586480e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.559+0000 7efd93e00640 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c561c0 0x5586480e6270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e27e2fa00afe7a0 server_cookie=84f33e8d67548fac in_seq=0 out_seq=0 2024-05-10T21:09:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.559+0000 7efd89600640 10 client.25214 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efd89600640 1 -- 172.21.15.50:0/4221048035 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558647fcffd0 con 0x558647c561c0 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efd89600640 10 client.25214 handle_command_reply: tid=0 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 client.25214 shutdown 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 msgr2=0x558648073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd94003fd0 tx=0x7efd940118d0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 shutdown_connections 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558647c561c0 0x5586480e6270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558647cc21b0 0x558648074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558647ca14d0 0x558648073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efd89600640 0 client.25214 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 --2- 172.21.15.50:0/4221048035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558647c992f0 0x558648073030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 >> 172.21.15.50:0/4221048035 conn(0x558647cb6300 msgr2=0x5586480634c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 shutdown_connections 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 1 -- 172.21.15.50:0/4221048035 wait complete. 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.567+0000 7efda9c0d740 20 client.25214 trim_cache size 0 max 16384 2024-05-10T21:09:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:09:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:09:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:09:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:09:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 msgr2=0x7efd800b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 0x7efd800b4d70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0a9410 tx=0x7efd8c009f10 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 msgr2=0x7efd9c103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9c0 tx=0x7efd8c00ee80 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 shutdown_connections 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efd800b2860 0x7efd800b4d70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd9c106b90 0x7efd9c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd9c109700 0x7efd9c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 --2- 172.21.15.50:0/4035090130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd9c1041a0 0x7efd9c103a10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 >> 172.21.15.50:0/4035090130 conn(0x7efd9c0fdb60 msgr2=0x7efd9c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 shutdown_connections 2024-05-10T21:09:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:00.568+0000 7efda1600640 1 -- 172.21.15.50:0/4035090130 wait complete. 2024-05-10T21:09:00.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.793+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24081a9e10 con 0x7f2408109870 2024-05-10T21:09:00.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.795+0000 7f2405600700 1 -- 172.21.15.50:0/3092282080 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22565 (secure 0 0 0) 0x7f24000ab6a0 con 0x7f2408109870 2024-05-10T21:09:00.796 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:09:00.796 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":82,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:04:56.878889+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:09:00.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 msgr2=0x7f23f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 0x7f23f00b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009c60 tx=0x7f23fc009260 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 msgr2=0x7f24081a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2400009790 tx=0x7f240000cc30 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 shutdown_connections 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f23f00b2900 0x7f23f00b4e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2408109870 0x7f24081a1990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24081045f0 0x7f240819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 --2- 172.21.15.50:0/3092282080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2408108f20 0x7f240819d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.798+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 >> 172.21.15.50:0/3092282080 conn(0x7f2408100040 msgr2=0x7f2408110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.799+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 shutdown_connections 2024-05-10T21:09:00.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:00.799+0000 7f240f800700 1 -- 172.21.15.50:0/3092282080 wait complete. 2024-05-10T21:09:01.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:00 smithi092 ceph-mon[34415]: pgmap v464: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:09:01.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:00 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3092282080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:09:01.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:00 smithi142 ceph-mon[34846]: pgmap v464: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:09:01.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:00 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3092282080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:09:01.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:00 smithi050 ceph-mon[29546]: pgmap v464: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:09:01.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:00 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3092282080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:09:01.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-10T21:09:02.116 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:03.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:02 smithi092 ceph-mon[34415]: pgmap v465: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:09:03.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:02 smithi142 ceph-mon[34846]: pgmap v465: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:09:03.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:02 smithi050 ceph-mon[29546]: pgmap v465: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:09:04.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:04 smithi050 ceph-mon[29546]: pgmap v466: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:09:05.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:04 smithi092 ceph-mon[34415]: pgmap v466: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:09:05.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:04 smithi142 ceph-mon[34846]: pgmap v466: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:09:05.598 INFO:teuthology.orchestra.run.smithi050.stdout:{"run_dir":"/var/run/ceph"} 2024-05-10T21:09:05.599 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-10T21:09:05.599 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2119 2024-05-10T21:09:05.599 DEBUG:tasks.quiescer:Yielding 2024-05-10T21:09:05.599 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T21:09:05.619 INFO:tasks.workunit:Pulling workunits from ref 7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T21:09:05.619 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T21:09:05.619 INFO:tasks.workunit:timeout=3h 2024-05-10T21:09:05.619 INFO:tasks.workunit:cleanup=True 2024-05-10T21:09:05.620 DEBUG:teuthology.orchestra.run.smithi050:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T21:09:05.621 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-10T21:09:05.674 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-10 21:05:06.368396749 +0000 2024-05-10T21:09:05.675 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-10 21:05:06.368396749 +0000 2024-05-10T21:09:05.675 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-10 21:08:16.467065296 +0000 2024-05-10T21:09:05.675 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-10 21:05:06.368396749 +0000 2024-05-10T21:09:05.675 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T21:09:05.675 DEBUG:teuthology.orchestra.run.smithi050:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T21:09:05.807 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b710596db3f44de0a4b72eff459db857656fe05 2024-05-10T21:09:05.860 INFO:tasks.workunit.client.0.smithi050.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T21:09:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:07.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:06 smithi142 ceph-mon[34846]: pgmap v467: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-10T21:09:07.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:06 smithi050 ceph-mon[29546]: pgmap v467: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-10T21:09:07.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:06 smithi092 ceph-mon[34415]: pgmap v467: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.363+0000 7f000da00700 1 -- 172.21.15.50:0/3704448286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 msgr2=0x7f000810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.363+0000 7f000da00700 1 --2- 172.21.15.50:0/3704448286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000810e3a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efff4009990 tx=0x7efff4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 -- 172.21.15.50:0/3704448286 shutdown_connections 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 --2- 172.21.15.50:0/3704448286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000810e8e0 0x7f0008110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 --2- 172.21.15.50:0/3704448286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0008105500 0x7f0008105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 --2- 172.21.15.50:0/3704448286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000810e3a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 -- 172.21.15.50:0/3704448286 >> 172.21.15.50:0/3704448286 conn(0x7f00080fedc0 msgr2=0x7f00081011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 -- 172.21.15.50:0/3704448286 shutdown_connections 2024-05-10T21:09:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.364+0000 7f000da00700 1 -- 172.21.15.50:0/3704448286 wait complete. 2024-05-10T21:09:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.365+0000 7f000da00700 1 Processor -- start 2024-05-10T21:09:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 -- start start 2024-05-10T21:09:08.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:08.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:08.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f000810e8e0 0x7f000819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:08.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0008114490 con 0x7f000810e8e0 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0008114310 con 0x7f0008105500 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f000da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0008114610 con 0x7f0008105e50 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f0006a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f0007400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f000810e8e0 0x7f000819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f0006000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f0006a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55484/0 (socket says 172.21.15.50:55484) 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.366+0000 7f0006a00700 1 -- 172.21.15.50:0/3453547653 learned_addr learned my addr 172.21.15.50:0/3453547653 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 -- 172.21.15.50:0/3453547653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 msgr2=0x7f000819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 -- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f000810e8e0 msgr2=0x7f000819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f000810e8e0 0x7f000819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 -- 172.21.15.50:0/3453547653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000819fcf0 con 0x7f0008105500 2024-05-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.367+0000 7f0006a00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9a0 tx=0x7efff800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.368+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800ca10 con 0x7f0008105500 2024-05-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.368+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff800cbb0 con 0x7f0008105500 2024-05-10T21:09:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.368+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8018680 con 0x7f0008105500 2024-05-10T21:09:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.368+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000819ffe0 con 0x7f0008105500 2024-05-10T21:09:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.368+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00081a7900 con 0x7f0008105500 2024-05-10T21:09:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.369+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00080512c0 con 0x7f0008105500 2024-05-10T21:09:08.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.372+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efff8010430 con 0x7f0008105500 2024-05-10T21:09:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.373+0000 7effff400700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 0x7effec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.374+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efff80e26f0 con 0x7f0008105500 2024-05-10T21:09:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.374+0000 7f0006000700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 0x7effec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:08.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.375+0000 7f0006000700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 0x7effec0b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0008102240 tx=0x7efff4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:08.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.376+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efff80aa3e0 con 0x7f0008105500 2024-05-10T21:09:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.662+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f000802e5f0 con 0x7f0008105500 2024-05-10T21:09:08.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.663+0000 7effff400700 1 -- 172.21.15.50:0/3453547653 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7efff80a9ce0 con 0x7f0008105500 2024-05-10T21:09:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:09:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 msgr2=0x7effec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 0x7effec0b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0008102240 tx=0x7efff4005fd0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 msgr2=0x7f000819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9a0 tx=0x7efff800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 shutdown_connections 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7effec0b2880 0x7effec0b4d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0008105500 0x7f000819ae20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f000810e8e0 0x7f000819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 --2- 172.21.15.50:0/3453547653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0008105e50 0x7f000819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:08.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 >> 172.21.15.50:0/3453547653 conn(0x7f00080fedc0 msgr2=0x7f00080ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:08.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 shutdown_connections 2024-05-10T21:09:08.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:08.667+0000 7f000da00700 1 -- 172.21.15.50:0/3453547653 wait complete. 2024-05-10T21:09:08.668 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:09:08.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:08 smithi050 ceph-mon[29546]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:08.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:08 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3453547653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:08 smithi142 ceph-mon[34846]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:08 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3453547653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:09.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:08 smithi092 ceph-mon[34415]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:09.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:08 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3453547653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:09.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T21:04:26.742871+0000', 'modified': '2024-05-10T21:08:17.695422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24539, 'mds_1': 24520, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/1464775005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1464775005}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1464775005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/924361476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 924361476}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 924361476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6837/1752993045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 1752993045}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1752993045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6839/3522449943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3522449943}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3522449943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.50:6841/908891204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6840', 'nonce': 908891204}, {'type': 'v1', 'addr': '172.21.15.50:6841', 'nonce': 908891204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6839/3200447854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 3200447854}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 3200447854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24520, 24539, 24502]} 2024-05-10T21:09:09.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T21:09:09.628 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T21:09:10.087 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:11.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:10 smithi050 ceph-mon[29546]: pgmap v469: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:11.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:10 smithi092 ceph-mon[34415]: pgmap v469: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:10 smithi142 ceph-mon[34846]: pgmap v469: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 -- 172.21.15.50:0/535560051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527810e700 msgr2=0x7f5278110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 --2- 172.21.15.50:0/535560051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527810e700 0x7f5278110be0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5270009990 tx=0x7f5270009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 -- 172.21.15.50:0/535560051 shutdown_connections 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 --2- 172.21.15.50:0/535560051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5278105290 0x7f52781056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 --2- 172.21.15.50:0/535560051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f527810e700 0x7f5278110be0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 --2- 172.21.15.50:0/535560051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105be0 0x7f527810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 -- 172.21.15.50:0/535560051 >> 172.21.15.50:0/535560051 conn(0x7f5278078ac0 msgr2=0x7f5278078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 -- 172.21.15.50:0/535560051 shutdown_connections 2024-05-10T21:09:12.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.634+0000 7f5277e00700 1 -- 172.21.15.50:0/535560051 wait complete. 2024-05-10T21:09:12.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.636+0000 7f5277e00700 1 Processor -- start 2024-05-10T21:09:12.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.636+0000 7f5277e00700 1 -- start start 2024-05-10T21:09:12.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 0x7f5278102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 0x7f52781034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 0x7f5278103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52781142e0 con 0x7f5278105be0 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114160 con 0x7f527810e700 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114460 con 0x7f5278105290 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5276a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 0x7f5278102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5276a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 0x7f5278102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39760/0 (socket says 172.21.15.50:39760) 2024-05-10T21:09:12.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5276a00700 1 -- 172.21.15.50:0/2389326421 learned_addr learned my addr 172.21.15.50:0/2389326421 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:12.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5276000700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 0x7f52781034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.637+0000 7f5277400700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 0x7f5278103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5276000700 1 -- 172.21.15.50:0/2389326421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 msgr2=0x7f5278102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5276000700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 0x7f5278102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5276000700 1 -- 172.21.15.50:0/2389326421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 msgr2=0x7f5278103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5276000700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 0x7f5278103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5276000700 1 -- 172.21.15.50:0/2389326421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52781a8ff0 con 0x7f5278105be0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.638+0000 7f5277400700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 0x7f5278103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f5276000700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 0x7f52781034c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f527ca00700 1 -- 172.21.15.50:0/2389326421 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004220 con 0x7f5278105be0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52781a92e0 con 0x7f5278105be0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f527ca00700 1 -- 172.21.15.50:0/2389326421 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640043c0 con 0x7f5278105be0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52781a9820 con 0x7f5278105be0 2024-05-10T21:09:12.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.639+0000 7f527ca00700 1 -- 172.21.15.50:0/2389326421 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264005020 con 0x7f5278105be0 2024-05-10T21:09:12.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.643+0000 7f527ca00700 1 -- 172.21.15.50:0/2389326421 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f52640051c0 con 0x7f5278105be0 2024-05-10T21:09:12.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.645+0000 7f528c9fbb80 1 Processor -- start 2024-05-10T21:09:12.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.645+0000 7f528c9fbb80 1 -- start start 2024-05-10T21:09:12.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 0x5569cc545cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 0x5569cc551de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569cc4679d0 con 0x5569cc546230 2024-05-10T21:09:12.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569cc405710 con 0x5569cc546640 2024-05-10T21:09:12.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569cc463990 con 0x5569cc3c7df0 2024-05-10T21:09:12.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f527ca00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 0x7f525c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f5276a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f5276a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34470/0 (socket says 172.21.15.50:34470) 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f5276a00700 1 -- 172.21.15.50:0/3972151026 learned_addr learned my addr 172.21.15.50:0/3972151026 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f527ca00700 1 -- 172.21.15.50:0/2389326421 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f52640e1ff0 con 0x7f5278105be0 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.646+0000 7f5277400700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 0x5569cc545cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276000700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 0x5569cc551de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5277400700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 0x7f525c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 -- 172.21.15.50:0/3972151026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 msgr2=0x5569cc545cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 0x5569cc545cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 -- 172.21.15.50:0/3972151026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 msgr2=0x5569cc551de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 0x5569cc551de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 -- 172.21.15.50:0/3972151026 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc3b4880 con 0x5569cc546230 2024-05-10T21:09:12.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5277400700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 0x5569cc545cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:12.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276000700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 0x5569cc551de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:12.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.647+0000 7f5276a00700 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52680094e0 tx=0x7f5268009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e8bd146e087d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.648+0000 7f526cc00700 1 -- 172.21.15.50:0/3972151026 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268019030 con 0x5569cc546230 2024-05-10T21:09:12.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.648+0000 7f5277400700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 0x7f525c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5569cc5414b0 tx=0x7f527000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.648+0000 7f526cc00700 1 -- 172.21.15.50:0/3972151026 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268004bb0 con 0x5569cc546230 2024-05-10T21:09:12.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.648+0000 7f526cc00700 1 -- 172.21.15.50:0/3972151026 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268005420 con 0x5569cc546230 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3972151026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 msgr2=0x5569cc54f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52680094e0 tx=0x7f5268009920 comp rx=0 tx=0).stop 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3972151026 shutdown_connections 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546640 0x5569cc551de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc3c7df0 0x5569cc545cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3972151026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.649+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3972151026 >> 172.21.15.50:0/3972151026 conn(0x5569cc406a30 msgr2=0x5569cc540b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.650+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3972151026 shutdown_connections 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.650+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3972151026 wait complete. 2024-05-10T21:09:12.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.650+0000 7f528c9fbb80 1 -- start start 2024-05-10T21:09:12.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 Processor -- start 2024-05-10T21:09:12.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569cc552750 con 0x5569cc546230 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569cc3cdad0 con 0x5569cc3c7df0 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569cc5528d0 con 0x5569cc546640 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.651+0000 7f528c9fbb80 10 client.0 _ll_register_callbacks cb 0x5569cc3fb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34476/0 (socket says 172.21.15.50:34476) 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 -- 172.21.15.50:0/3213501539 learned_addr learned my addr 172.21.15.50:0/3213501539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55534/0 (socket says 172.21.15.50:55534) 2024-05-10T21:09:12.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5277400700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 -- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 msgr2=0x5569cc54fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 -- 172.21.15.50:0/3213501539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 msgr2=0x5569cc54f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276000700 1 -- 172.21.15.50:0/3213501539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc42d3d0 con 0x5569cc546230 2024-05-10T21:09:12.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.652+0000 7f5276a00700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5276000700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f526400c5b0 tx=0x7f5264091330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd37702fd0727147 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5261600700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526401c070 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5268004bb0 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f52640054b0 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52640a9640 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f528c9fbb80 10 client.15117 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569cc55d400 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.653+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268004bb0 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f52640b2010 con 0x5569cc546230 2024-05-10T21:09:12.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 20 client.15117 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc55fac0 0x5569cc560f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 4 client.15117 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569cc512020 con 0x5569cc55fac0 2024-05-10T21:09:12.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.654+0000 7f5276a00700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc55fac0 0x5569cc560f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.655+0000 7f5276a00700 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc55fac0 0x5569cc560f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72e3a60633c4cdd3 server_cookie=900b66b578dcf043 in_seq=0 out_seq=0 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.655+0000 7f5261600700 10 client.15117 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f5261600700 1 -- 172.21.15.50:0/3213501539 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569cc512020 con 0x5569cc55fac0 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f5261600700 10 client.15117 handle_command_reply: tid=0 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 client.15117 shutdown 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 msgr2=0x5569cc54f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f526400c5b0 tx=0x7f5264091330 comp rx=0 tx=0).stop 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 shutdown_connections 2024-05-10T21:09:12.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc55fac0 0x5569cc560f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc54f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/3213501539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc54f8d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 >> 172.21.15.50:0/3213501539 conn(0x5569cc406a30 msgr2=0x5569cc5446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.656+0000 7f5261600700 0 client.15117 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.657+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 shutdown_connections 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.657+0000 7f528c9fbb80 1 -- 172.21.15.50:0/3213501539 wait complete. 2024-05-10T21:09:12.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.657+0000 7f528c9fbb80 20 client.15117 trim_cache size 0 max 16384 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.680+0000 7f528c9fbb80 1 Processor -- start 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.680+0000 7f528c9fbb80 1 -- start start 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc3c7df0 0x5569cc54e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 0x5569cc55b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569cc55f270 con 0x5569cc3c7df0 2024-05-10T21:09:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569cc55f940 con 0x5569cc546230 2024-05-10T21:09:12.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569cc560010 con 0x5569cc546640 2024-05-10T21:09:12.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5277400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5276000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 0x5569cc55b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5276000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 0x5569cc55b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55544/0 (socket says 172.21.15.50:55544) 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5277400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39794/0 (socket says 172.21.15.50:39794) 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5276000700 1 -- 172.21.15.50:0/465355853 learned_addr learned my addr 172.21.15.50:0/465355853 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.681+0000 7f5276a00700 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc3c7df0 0x5569cc54e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 -- 172.21.15.50:0/465355853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 msgr2=0x5569cc55b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 0x5569cc55b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 -- 172.21.15.50:0/465355853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc3c7df0 msgr2=0x5569cc54e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc3c7df0 0x5569cc54e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 -- 172.21.15.50:0/465355853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc552280 con 0x5569cc546640 2024-05-10T21:09:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5277400700 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5270005f50 tx=0x7f527001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f91235da545ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5263e00700 1 -- 172.21.15.50:0/465355853 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000fba0 con 0x5569cc546640 2024-05-10T21:09:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5263e00700 1 -- 172.21.15.50:0/465355853 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527000fd40 con 0x5569cc546640 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.682+0000 7f5263e00700 1 -- 172.21.15.50:0/465355853 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270029680 con 0x5569cc546640 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.683+0000 7f528c9fbb80 1 -- 172.21.15.50:0/465355853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 msgr2=0x5569cc54f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.683+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5270005f50 tx=0x7f527001a040 comp rx=0 tx=0).stop 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 -- 172.21.15.50:0/465355853 shutdown_connections 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc54f900 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc3c7df0 0x5569cc54e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/465355853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc546230 0x5569cc55b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 -- 172.21.15.50:0/465355853 >> 172.21.15.50:0/465355853 conn(0x5569cc406a30 msgr2=0x5569cc55abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 -- 172.21.15.50:0/465355853 shutdown_connections 2024-05-10T21:09:12.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 -- 172.21.15.50:0/465355853 wait complete. 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.684+0000 7f528c9fbb80 1 -- start start 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.685+0000 7f528c9fbb80 1 Processor -- start 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc564090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc564b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569cc560db0 con 0x5569cc546230 2024-05-10T21:09:12.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569cc5606e0 con 0x5569cc3c7df0 2024-05-10T21:09:12.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569cc561480 con 0x5569cc546640 2024-05-10T21:09:12.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f528c9fbb80 10 client.0 _ll_register_callbacks cb 0x5569cc3fb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f5276a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc564090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f5276000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f5276000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34498/0 (socket says 172.21.15.50:34498) 2024-05-10T21:09:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f5276000700 1 -- 172.21.15.50:0/4144108674 learned_addr learned my addr 172.21.15.50:0/4144108674 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.686+0000 7f5277400700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc564b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 -- 172.21.15.50:0/4144108674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 msgr2=0x5569cc564090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc564090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 -- 172.21.15.50:0/4144108674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 msgr2=0x5569cc5645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 -- 172.21.15.50:0/4144108674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc547060 con 0x5569cc546640 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5276000700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5277400700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc564b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5270005f50 tx=0x7f5270005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4e6f30059266ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5261600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000fba0 con 0x5569cc546640 2024-05-10T21:09:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f527002f040 con 0x5569cc546640 2024-05-10T21:09:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.687+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527000fd40 con 0x5569cc546640 2024-05-10T21:09:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5270003db0 con 0x5569cc546640 2024-05-10T21:09:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f528c9fbb80 10 client.25234 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569cc5458a0 con 0x5569cc546640 2024-05-10T21:09:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527000fd40 con 0x5569cc546640 2024-05-10T21:09:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5270033770 con 0x5569cc546640 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f528c9fbb80 10 client.25234 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f528c9fbb80 10 client.25234 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.688+0000 7f528c9fbb80 20 client.25234 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc5cce40 0x5569cc5ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f528c9fbb80 4 client.25234 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:09:12.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "status"}) -- 0x5569cc4753b0 con 0x5569cc5cce40 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f5276a00700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc5cce40 0x5569cc5ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f5276a00700 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc5cce40 0x5569cc5ce2b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e02cbdd9b218e574 server_cookie=fdc49ea5780862c7 in_seq=0 out_seq=0 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.689+0000 7f5261600700 10 client.25234 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f5261600700 1 -- 172.21.15.50:0/4144108674 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5569cc4753b0 con 0x5569cc5cce40 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f5261600700 10 client.25234 handle_command_reply: tid=0 2024-05-10T21:09:12.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 client.25234 shutdown 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 msgr2=0x5569cc564b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc564b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5270005f50 tx=0x7f5270005ee0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 shutdown_connections 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x5569cc5cce40 0x5569cc5ce2b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f5261600700 0 client.25234 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:12.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569cc3c7df0 0x5569cc564090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569cc546640 0x5569cc564b10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 --2- 172.21.15.50:0/4144108674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569cc546230 0x5569cc5645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 >> 172.21.15.50:0/4144108674 conn(0x5569cc406a30 msgr2=0x5569cc5506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.690+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 shutdown_connections 2024-05-10T21:09:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.691+0000 7f528c9fbb80 1 -- 172.21.15.50:0/4144108674 wait complete. 2024-05-10T21:09:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.691+0000 7f528c9fbb80 20 client.25234 trim_cache size 0 max 16384 2024-05-10T21:09:12.696 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:12.696 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:09:12.696 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24502, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 268.88916302500002, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 82, 2024-05-10T21:09:12.697 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 319.810710884 2024-05-10T21:09:12.698 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 msgr2=0x7f525c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 0x7f525c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5569cc5414b0 tx=0x7f527000b260 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 msgr2=0x7f52781034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 0x7f52781034c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400ed00 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 shutdown_connections 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f525c0b2900 0x7f525c0b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f527810e700 0x7f5278103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5278105290 0x7f5278102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 --2- 172.21.15.50:0/2389326421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5278105be0 0x7f52781034c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 >> 172.21.15.50:0/2389326421 conn(0x7f5278078ac0 msgr2=0x7f52780fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 shutdown_connections 2024-05-10T21:09:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:12.692+0000 7f5277e00700 1 -- 172.21.15.50:0/2389326421 wait complete. 2024-05-10T21:09:13.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:12 smithi050 ceph-mon[29546]: pgmap v470: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:13.175 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:09:13.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:12 smithi092 ceph-mon[34415]: pgmap v470: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:12 smithi142 ceph-mon[34846]: pgmap v470: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:09:13.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.a status 2024-05-10T21:09:13.759 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 -- 172.21.15.50:0/476945876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 msgr2=0x7f86fc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 --2- 172.21.15.50:0/476945876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc104960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002a3f0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 -- 172.21.15.50:0/476945876 shutdown_connections 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 --2- 172.21.15.50:0/476945876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc109ba0 0x7f86fc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 --2- 172.21.15.50:0/476945876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc104ea0 0x7f86fc1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 --2- 172.21.15.50:0/476945876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc104960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.818+0000 7f8703a00640 1 -- 172.21.15.50:0/476945876 >> 172.21.15.50:0/476945876 conn(0x7f86fc0fee50 msgr2=0x7f86fc101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.819+0000 7f8703a00640 1 -- 172.21.15.50:0/476945876 shutdown_connections 2024-05-10T21:09:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.819+0000 7f8703a00640 1 -- 172.21.15.50:0/476945876 wait complete. 2024-05-10T21:09:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 Processor -- start 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 -- start start 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 0x7f86fc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc109ba0 0x7f86fc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10f7b0 con 0x7f86fc103c60 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10f630 con 0x7f86fc109ba0 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.820+0000 7f8703a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10f930 con 0x7f86fc104ea0 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f8701000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 0x7f86fc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f8701000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34510/0 (socket says 172.21.15.50:34510) 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f8701000640 1 -- 172.21.15.50:0/32931911 learned_addr learned my addr 172.21.15.50:0/32931911 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f8701a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc109ba0 0x7f86fc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 -- 172.21.15.50:0/32931911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc109ba0 msgr2=0x7f86fc1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc109ba0 0x7f86fc1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 -- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 msgr2=0x7f86fc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.821+0000 7f86fbe00640 1 -- 172.21.15.50:0/32931911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc1a1ec0 con 0x7f86fc104ea0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f86fbe00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 0x7f86fc19a8c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9f0 tx=0x7f86e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f86f9600640 1 -- 172.21.15.50:0/32931911 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e800cd80 con 0x7f86fc104ea0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f86f9600640 1 -- 172.21.15.50:0/32931911 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e80035f0 con 0x7f86fc104ea0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1a21b0 con 0x7f86fc104ea0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f86f9600640 1 -- 172.21.15.50:0/32931911 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8010670 con 0x7f86fc104ea0 2024-05-10T21:09:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.822+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a2750 con 0x7f86fc104ea0 2024-05-10T21:09:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 Processor -- start 2024-05-10T21:09:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 -- start start 2024-05-10T21:09:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dc5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.826+0000 7f870a9d0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5647b0d25550 con 0x5647b0a1f1b0 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f870a9d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5647b0ccabc0 con 0x5647b09f62f0 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f870a9d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5647b0d21510 con 0x5647b09fe4d0 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34526/0 (socket says 172.21.15.50:34526) 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 -- 172.21.15.50:0/1879268954 learned_addr learned my addr 172.21.15.50:0/1879268954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701000640 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dc5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 -- 172.21.15.50:0/1879268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 msgr2=0x5647b0dcc990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:09:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcc990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 -- 172.21.15.50:0/1879268954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 msgr2=0x5647b0dc5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dc5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.827+0000 7f8701a00640 1 -- 172.21.15.50:0/1879268954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b0a37210 con 0x5647b0a1f1b0 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.828+0000 7f8701a00640 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fd0 tx=0x7f86f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1701e0e61ca10dbc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.828+0000 7f86ee000640 1 -- 172.21.15.50:0/1879268954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4004d20 con 0x5647b0a1f1b0 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.828+0000 7f86ee000640 1 -- 172.21.15.50:0/1879268954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4004050 con 0x5647b0a1f1b0 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.828+0000 7f86ee000640 1 -- 172.21.15.50:0/1879268954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4005260 con 0x5647b0a1f1b0 2024-05-10T21:09:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.828+0000 7f86f9600640 1 -- 172.21.15.50:0/32931911 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86e80108d0 con 0x7f86fc104ea0 2024-05-10T21:09:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.829+0000 7f870a9d0740 1 -- 172.21.15.50:0/1879268954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 msgr2=0x5647b0dc5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.829+0000 7f870a9d0740 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fd0 tx=0x7f86f40118d0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 -- 172.21.15.50:0/1879268954 shutdown_connections 2024-05-10T21:09:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dc5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 --2- 172.21.15.50:0/1879268954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dc5510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 -- 172.21.15.50:0/1879268954 >> 172.21.15.50:0/1879268954 conn(0x5647b0a13300 msgr2=0x5647b0dbca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 -- 172.21.15.50:0/1879268954 shutdown_connections 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 -- 172.21.15.50:0/1879268954 wait complete. 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f86f9600640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 0x7f86e40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.830+0000 7f870a9d0740 1 -- start start 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f86f9600640 1 -- 172.21.15.50:0/32931911 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f86e80e2450 con 0x7f86fc104ea0 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f8701a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 0x7f86e40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f870a9d0740 1 Processor -- start 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dcf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.831+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f870a9d0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5647b0dcd3e0 con 0x5647b0a1f1b0 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f870a9d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5647b0cde660 con 0x5647b09f62f0 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f870a9d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5647b0dcd560 con 0x5647b09fe4d0 2024-05-10T21:09:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f870a9d0740 10 client.0 _ll_register_callbacks cb 0x5647b0ca8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f8701000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dcf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f8701000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dcf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55584/0 (socket says 172.21.15.50:55584) 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f8701000640 1 -- 172.21.15.50:0/813327666 learned_addr learned my addr 172.21.15.50:0/813327666 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f86fbe00640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f8701a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 0x7f86e40b4d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009de0 tx=0x7f86f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f8701a00640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f86fbe00640 1 -- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 msgr2=0x5647b0dcf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f86fbe00640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dcf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f86fbe00640 1 -- 172.21.15.50:0/813327666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 msgr2=0x5647b0dd64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.832+0000 7f86fbe00640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86fbe00640 1 -- 172.21.15.50:0/813327666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b0b03620 con 0x5647b09fe4d0 2024-05-10T21:09:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86fbe00640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcf560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9c0 tx=0x7f86e80a9550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4517e9bb3618c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80addf0 con 0x5647b09fe4d0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f86e80a6030 con 0x5647b09fe4d0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f4004050 con 0x5647b09fe4d0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86e80b28a0 con 0x5647b09fe4d0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.833+0000 7f870a9d0740 10 client.25243 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647b0dd9440 con 0x5647b09fe4d0 2024-05-10T21:09:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f4004050 con 0x5647b09fe4d0 2024-05-10T21:09:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f86e80b2410 con 0x5647b09fe4d0 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 10 client.25243 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 10 client.25243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 20 client.25243 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09ddd80 0x5647b0ddcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 4 client.25243 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.834+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b0d33160 con 0x5647b09ddd80 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.835+0000 7f8701000640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09ddd80 0x5647b0ddcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.835+0000 7f8701000640 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09ddd80 0x5647b0ddcad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6587bd57cf22322e server_cookie=edf9d4ae22104a89 in_seq=0 out_seq=0 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.835+0000 7f86e2a00640 10 client.25243 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.836+0000 7f86e2a00640 1 -- 172.21.15.50:0/813327666 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647b0d33160 con 0x5647b09ddd80 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.836+0000 7f86e2a00640 10 client.25243 handle_command_reply: tid=0 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.836+0000 7f870a9d0740 1 client.25243 shutdown 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 msgr2=0x5647b0dcf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcf560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9c0 tx=0x7f86e80a9550 comp rx=0 tx=0).stop 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 shutdown_connections 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09ddd80 0x5647b0ddcad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09fe4d0 0x5647b0dcf560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f86e2a00640 0 client.25243 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 --2- 172.21.15.50:0/813327666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09f62f0 0x5647b0dcf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 >> 172.21.15.50:0/813327666 conn(0x5647b0a13300 msgr2=0x5647b0dbba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 shutdown_connections 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 1 -- 172.21.15.50:0/813327666 wait complete. 2024-05-10T21:09:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.837+0000 7f870a9d0740 20 client.25243 trim_cache size 0 max 16384 2024-05-10T21:09:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 Processor -- start 2024-05-10T21:09:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 -- start start 2024-05-10T21:09:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09fe4d0 0x5647b0dd63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b0a1f1b0 0x5647b0dce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ab920 con 0x5647b09f62f0 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a91f0 con 0x5647b09fe4d0 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.872+0000 7f870a9d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a7040 con 0x5647b0a1f1b0 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09fe4d0 0x5647b0dd63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46204/0 (socket says 172.21.15.50:46204) 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 -- 172.21.15.50:0/2877709184 learned_addr learned my addr 172.21.15.50:0/2877709184 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701a00640 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b0a1f1b0 0x5647b0dce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 -- 172.21.15.50:0/2877709184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b0a1f1b0 msgr2=0x5647b0dce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b0a1f1b0 0x5647b0dce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 -- 172.21.15.50:0/2877709184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09fe4d0 msgr2=0x5647b0dd63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09fe4d0 0x5647b0dd63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.873+0000 7f8701000640 1 -- 172.21.15.50:0/2877709184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b0db80b0 con 0x5647b09f62f0 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.874+0000 7f8701000640 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f86f002a9b0 tx=0x7f86f0004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c4e949e44cb4c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.874+0000 7f86ed600640 1 -- 172.21.15.50:0/2877709184 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0035040 con 0x5647b09f62f0 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.874+0000 7f86ed600640 1 -- 172.21.15.50:0/2877709184 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0046ab0 con 0x5647b09f62f0 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.874+0000 7f86ed600640 1 -- 172.21.15.50:0/2877709184 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00315a0 con 0x5647b09f62f0 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 -- 172.21.15.50:0/2877709184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 msgr2=0x5647b0dc0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f86f002a9b0 tx=0x7f86f0004510 comp rx=0 tx=0).stop 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 -- 172.21.15.50:0/2877709184 shutdown_connections 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b0a1f1b0 0x5647b0dce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b09fe4d0 0x5647b0dd63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 --2- 172.21.15.50:0/2877709184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09f62f0 0x5647b0dc0020 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 -- 172.21.15.50:0/2877709184 >> 172.21.15.50:0/2877709184 conn(0x5647b0a13300 msgr2=0x5647b0dbf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 -- 172.21.15.50:0/2877709184 shutdown_connections 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.875+0000 7f870a9d0740 1 -- 172.21.15.50:0/2877709184 wait complete. 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.876+0000 7f870a9d0740 1 -- start start 2024-05-10T21:09:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.876+0000 7f870a9d0740 1 Processor -- start 2024-05-10T21:09:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09f62f0 0x5647b0dd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a76b0 con 0x5647b09fe4d0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a8b20 con 0x5647b0a1f1b0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a7d80 con 0x5647b09f62f0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f870a9d0740 10 client.0 _ll_register_callbacks cb 0x7f86e80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f86fbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46208/0 (socket says 172.21.15.50:46208) 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f8701a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f86fbe00640 1 -- 172.21.15.50:0/4017273828 learned_addr learned my addr 172.21.15.50:0/4017273828 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.877+0000 7f8701000640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09f62f0 0x5647b0dd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 -- 172.21.15.50:0/4017273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09f62f0 msgr2=0x5647b0dd3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09f62f0 0x5647b0dd3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 -- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 msgr2=0x5647b0dd4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 -- 172.21.15.50:0/4017273828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b0dd5440 con 0x5647b09fe4d0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f8701a00640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86fbe00640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cfd0 tx=0x7f86e8009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57f242d664632f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.878+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80addf0 con 0x5647b09fe4d0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f86e80a6030 con 0x5647b09fe4d0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647b0d2cfd0 con 0x5647b09fe4d0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f86e80a9390 con 0x5647b09fe4d0 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f870a9d0740 10 client.15141 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647b0dd6770 con 0x5647b09fe4d0 2024-05-10T21:09:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5647b0d2cfd0 con 0x5647b09fe4d0 2024-05-10T21:09:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.879+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f86e8088070 con 0x5647b09fe4d0 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 10 client.15141 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 10 client.15141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 20 client.15141 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09b31c0 0x5647b0e475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 4 client.15141 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647b0ca84b0 con 0x5647b09b31c0 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.880+0000 7f8701000640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09b31c0 0x5647b0e475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.881+0000 7f8701000640 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09b31c0 0x5647b0e475c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772dae4dd3680f0a server_cookie=23cf4237e8cc3292 in_seq=0 out_seq=0 2024-05-10T21:09:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.881+0000 7f86e2a00640 10 client.15141 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.890+0000 7f86e2a00640 1 -- 172.21.15.50:0/4017273828 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647b0ca84b0 con 0x5647b09b31c0 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f86e2a00640 10 client.15141 handle_command_reply: tid=0 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 client.15141 shutdown 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 msgr2=0x5647b0dd3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cfd0 tx=0x7f86e8009540 comp rx=0 tx=0).stop 2024-05-10T21:09:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 shutdown_connections 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5647b09b31c0 0x5647b0e475c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5647b0a1f1b0 0x5647b0dd4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5647b09fe4d0 0x5647b0dd3fa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 --2- 172.21.15.50:0/4017273828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b09f62f0 0x5647b0dd3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 >> 172.21.15.50:0/4017273828 conn(0x5647b0a13300 msgr2=0x5647b0dc00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f86e2a00640 0 client.15141 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 shutdown_connections 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.891+0000 7f870a9d0740 1 -- 172.21.15.50:0/4017273828 wait complete. 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.892+0000 7f870a9d0740 20 client.15141 trim_cache size 0 max 16384 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:09:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:09:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:09:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:09:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 msgr2=0x7f86e40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 0x7f86e40b4d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009de0 tx=0x7f86f4005fd0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 msgr2=0x7f86fc19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 0x7f86fc19a8c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9f0 tx=0x7f86e800eeb0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 shutdown_connections 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f86e40b27f0 0x7f86e40b4d00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86fc104ea0 0x7f86fc19a8c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86fc109ba0 0x7f86fc1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 --2- 172.21.15.50:0/32931911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86fc103c60 0x7f86fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 >> 172.21.15.50:0/32931911 conn(0x7f86fc0fee50 msgr2=0x7f86fc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 shutdown_connections 2024-05-10T21:09:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:13.893+0000 7f8703a00640 1 -- 172.21.15.50:0/32931911 wait complete. 2024-05-10T21:09:15.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:14 smithi050 ceph-mon[29546]: pgmap v471: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:15.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:14 smithi092 ceph-mon[34415]: pgmap v471: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:14 smithi142 ceph-mon[34846]: pgmap v471: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 -- 172.21.15.50:0/3713944303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c108f10 msgr2=0x7f006c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 --2- 172.21.15.50:0/3713944303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c108f10 0x7f006c109320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f00540099c0 tx=0x7f0054009cd0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 -- 172.21.15.50:0/3713944303 shutdown_connections 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 --2- 172.21.15.50:0/3713944303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c1045e0 0x7f006c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 --2- 172.21.15.50:0/3713944303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c109860 0x7f006c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 --2- 172.21.15.50:0/3713944303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c108f10 0x7f006c109320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.295+0000 7f0072800700 1 -- 172.21.15.50:0/3713944303 >> 172.21.15.50:0/3713944303 conn(0x7f006c0ffed0 msgr2=0x7f006c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.296+0000 7f0072800700 1 -- 172.21.15.50:0/3713944303 shutdown_connections 2024-05-10T21:09:16.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.296+0000 7f0072800700 1 -- 172.21.15.50:0/3713944303 wait complete. 2024-05-10T21:09:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.297+0000 7f0072800700 1 Processor -- start 2024-05-10T21:09:16.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.297+0000 7f0072800700 1 -- start start 2024-05-10T21:09:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c108f10 0x7f006c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c109860 0x7f006c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f006c114470 con 0x7f006c1045e0 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1142f0 con 0x7f006c109860 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f0072800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1145f0 con 0x7f006c108f10 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f006b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f006b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46234/0 (socket says 172.21.15.50:46234) 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.298+0000 7f006b400700 1 -- 172.21.15.50:0/3469172317 learned_addr learned my addr 172.21.15.50:0/3469172317 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006aa00700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c108f10 0x7f006c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006be00700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c109860 0x7f006c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 -- 172.21.15.50:0/3469172317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c108f10 msgr2=0x7f006c077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c108f10 0x7f006c077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 -- 172.21.15.50:0/3469172317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c109860 msgr2=0x7f006c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c109860 0x7f006c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 -- 172.21.15.50:0/3469172317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c1a91a0 con 0x7f006c1045e0 2024-05-10T21:09:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.299+0000 7f006b400700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f005c00eb30 tx=0x7f005c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.300+0000 7f0063e00700 1 -- 172.21.15.50:0/3469172317 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c00cc90 con 0x7f006c1045e0 2024-05-10T21:09:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.300+0000 7f0063e00700 1 -- 172.21.15.50:0/3469172317 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c0044d0 con 0x7f006c1045e0 2024-05-10T21:09:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.300+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c1a9490 con 0x7f006c1045e0 2024-05-10T21:09:16.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.300+0000 7f0063e00700 1 -- 172.21.15.50:0/3469172317 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c010430 con 0x7f006c1045e0 2024-05-10T21:09:16.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.300+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a99a0 con 0x7f006c1045e0 2024-05-10T21:09:16.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.304+0000 7f0063e00700 1 -- 172.21.15.50:0/3469172317 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f005c003ea0 con 0x7f006c1045e0 2024-05-10T21:09:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.305+0000 7f008143fb80 1 Processor -- start 2024-05-10T21:09:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 -- start start 2024-05-10T21:09:16.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b21d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b217710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b218060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55693b139a20 con 0x55693b217c50 2024-05-10T21:09:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55693b0d7760 con 0x55693b099e40 2024-05-10T21:09:16.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.306+0000 7f008143fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55693b1359e0 con 0x55693b21dbf0 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b21d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b217710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b21d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46222/0 (socket says 172.21.15.50:46222) 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006be00700 1 -- 172.21.15.50:0/1816072051 learned_addr learned my addr 172.21.15.50:0/1816072051 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f0063e00700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 0x7f00500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006aa00700 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b218060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006be00700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 0x7f00500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f0063e00700 1 -- 172.21.15.50:0/3469172317 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f005c0e24e0 con 0x7f006c1045e0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 -- 172.21.15.50:0/1816072051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 msgr2=0x55693b21d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b21d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 -- 172.21.15.50:0/1816072051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 msgr2=0x55693b218060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b218060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.307+0000 7f006b400700 1 -- 172.21.15.50:0/1816072051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693b0868d0 con 0x55693b21dbf0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.308+0000 7f006b400700 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b217710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f005c0a6d40 tx=0x7f005c003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d561e3bec5dc0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.308+0000 7f0060c00700 1 -- 172.21.15.50:0/1816072051 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0a98a0 con 0x55693b21dbf0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.308+0000 7f0060c00700 1 -- 172.21.15.50:0/1816072051 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c0a95e0 con 0x55693b21dbf0 2024-05-10T21:09:16.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.308+0000 7f0060c00700 1 -- 172.21.15.50:0/1816072051 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0a5260 con 0x55693b21dbf0 2024-05-10T21:09:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 -- 172.21.15.50:0/1816072051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 msgr2=0x55693b217710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f006be00700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 0x7f00500b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55693b218a30 tx=0x7f0064004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b217710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f005c0a6d40 tx=0x7f005c003940 comp rx=0 tx=0).stop 2024-05-10T21:09:16.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 -- 172.21.15.50:0/1816072051 shutdown_connections 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b21d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b218060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 --2- 172.21.15.50:0/1816072051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b217710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 -- 172.21.15.50:0/1816072051 >> 172.21.15.50:0/1816072051 conn(0x55693b0d8a80 msgr2=0x55693b212790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.309+0000 7f008143fb80 1 -- 172.21.15.50:0/1816072051 shutdown_connections 2024-05-10T21:09:16.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.310+0000 7f008143fb80 1 -- 172.21.15.50:0/1816072051 wait complete. 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.310+0000 7f008143fb80 1 -- start start 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 Processor -- start 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b221320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b221860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b221da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55693b224660 con 0x55693b217c50 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55693b09fb20 con 0x55693b099e40 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55693b2247e0 con 0x55693b21dbf0 2024-05-10T21:09:16.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.311+0000 7f008143fb80 10 client.0 _ll_register_callbacks cb 0x55693b0cd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b221da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b221860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b221320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b221da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52880/0 (socket says 172.21.15.50:52880) 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006be00700 1 -- 172.21.15.50:0/4109405977 learned_addr learned my addr 172.21.15.50:0/4109405977 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 -- 172.21.15.50:0/4109405977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 msgr2=0x55693b221da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b221da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 -- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 msgr2=0x55693b221320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b221320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006aa00700 1 -- 172.21.15.50:0/4109405977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693b0ff420 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.312+0000 7f006b400700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b221320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f006aa00700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b221860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f005400b580 tx=0x7f0054005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0b90491c2f6a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f0059600700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540039c0 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005c0a95e0 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f0054003b60 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0054009e60 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f008143fb80 10 client.15153 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.313+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55693b22f1d0 con 0x55693b217c50 2024-05-10T21:09:16.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c0a95e0 con 0x55693b217c50 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f00540246a0 con 0x55693b217c50 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 10 client.15153 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 10 client.15153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 20 client.15153 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b231810 0x55693b233c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 4 client.15153 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.314+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693b1e4070 con 0x55693b231810 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.315+0000 7f006b400700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b231810 0x55693b233c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.315+0000 7f006b400700 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b231810 0x55693b233c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aae59f30e6e0852 server_cookie=7c9f6c371a77769f in_seq=0 out_seq=0 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.315+0000 7f0059600700 10 client.15153 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f0059600700 1 -- 172.21.15.50:0/4109405977 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55693b1e4070 con 0x55693b231810 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f0059600700 10 client.15153 handle_command_reply: tid=0 2024-05-10T21:09:16.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 client.15153 shutdown 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 msgr2=0x55693b221860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b221860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f005400b580 tx=0x7f0054005790 comp rx=0 tx=0).stop 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 shutdown_connections 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b231810 0x55693b233c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b099e40 0x55693b221320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b21dbf0 0x55693b221da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 --2- 172.21.15.50:0/4109405977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b221860 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f0059600700 0 client.15153 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.316+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 >> 172.21.15.50:0/4109405977 conn(0x55693b0d8a80 msgr2=0x55693b216700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.317+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 shutdown_connections 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.317+0000 7f008143fb80 1 -- 172.21.15.50:0/4109405977 wait complete. 2024-05-10T21:09:16.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.317+0000 7f008143fb80 20 client.15153 trim_cache size 0 max 16384 2024-05-10T21:09:16.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.340+0000 7f008143fb80 1 Processor -- start 2024-05-10T21:09:16.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.340+0000 7f008143fb80 1 -- start start 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b217710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b21ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b3520 con 0x55693b217c50 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b0db0 con 0x55693b21dbf0 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f008143fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1aec80 con 0x55693b099e40 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f006aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b21ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f006b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b217710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f006be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f006be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46246/0 (socket says 172.21.15.50:46246) 2024-05-10T21:09:16.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.341+0000 7f006be00700 1 -- 172.21.15.50:0/2558931098 learned_addr learned my addr 172.21.15.50:0/2558931098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:16.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 -- 172.21.15.50:0/2558931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 msgr2=0x55693b217710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b217710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 -- 172.21.15.50:0/2558931098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 msgr2=0x55693b21ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b21ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 -- 172.21.15.50:0/2558931098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693b21f890 con 0x55693b21dbf0 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f006be00700 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0064017860 tx=0x7f0064009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da107ea079942c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f005be00700 1 -- 172.21.15.50:0/2558931098 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064028030 con 0x55693b21dbf0 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f005be00700 1 -- 172.21.15.50:0/2558931098 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f006401d070 con 0x55693b21dbf0 2024-05-10T21:09:16.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.342+0000 7f005be00700 1 -- 172.21.15.50:0/2558931098 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064024d30 con 0x55693b21dbf0 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 -- 172.21.15.50:0/2558931098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 msgr2=0x55693b2219f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0064017860 tx=0x7f0064009920 comp rx=0 tx=0).stop 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 -- 172.21.15.50:0/2558931098 shutdown_connections 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b21dbf0 0x55693b2219f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b217710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 --2- 172.21.15.50:0/2558931098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b217c50 0x55693b21ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.343+0000 7f008143fb80 1 -- 172.21.15.50:0/2558931098 >> 172.21.15.50:0/2558931098 conn(0x55693b0d8a80 msgr2=0x55693b2212e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.344+0000 7f008143fb80 1 -- 172.21.15.50:0/2558931098 shutdown_connections 2024-05-10T21:09:16.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.344+0000 7f008143fb80 1 -- 172.21.15.50:0/2558931098 wait complete. 2024-05-10T21:09:16.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.344+0000 7f008143fb80 1 -- start start 2024-05-10T21:09:16.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.344+0000 7f008143fb80 1 Processor -- start 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b217c50 0x55693b229f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b21dbf0 0x55693b226a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1af2d0 con 0x55693b21dbf0 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b06e0 con 0x55693b217c50 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1af940 con 0x55693b099e40 2024-05-10T21:09:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f008143fb80 10 client.0 _ll_register_callbacks cb 0x55693b0cd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f006b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f006be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b21dbf0 0x55693b226a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f006b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52884/0 (socket says 172.21.15.50:52884) 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f006b400700 1 -- 172.21.15.50:0/3247777354 learned_addr learned my addr 172.21.15.50:0/3247777354 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.345+0000 7f006aa00700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b217c50 0x55693b229f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 -- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b217c50 msgr2=0x55693b229f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b217c50 0x55693b229f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 -- 172.21.15.50:0/3247777354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b21dbf0 msgr2=0x55693b226a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b21dbf0 0x55693b226a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 -- 172.21.15.50:0/3247777354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693b226ff0 con 0x55693b099e40 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f006b400700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f005c0a6d40 tx=0x7f005c00cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bf920bf11d057d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f0059600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.346+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0a9240 con 0x55693b099e40 2024-05-10T21:09:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006401d070 con 0x55693b099e40 2024-05-10T21:09:16.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f005c003680 con 0x55693b099e40 2024-05-10T21:09:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f005c0a9420 con 0x55693b099e40 2024-05-10T21:09:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 10 client.25267 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55693b29c7e0 con 0x55693b099e40 2024-05-10T21:09:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f006401d070 con 0x55693b099e40 2024-05-10T21:09:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f005c0a5d60 con 0x55693b099e40 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 10 client.25267 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 10 client.25267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 20 client.25267 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b29c9b0 0x55693b29edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 4 client.25267 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.347+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55693b147400 con 0x55693b29c9b0 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.348+0000 7f006aa00700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b29c9b0 0x55693b29edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.348+0000 7f006aa00700 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b29c9b0 0x55693b29edd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26cd78e1dc4100c8 server_cookie=7b7a1a88ace18d81 in_seq=0 out_seq=0 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.348+0000 7f0059600700 10 client.25267 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f0059600700 1 -- 172.21.15.50:0/3247777354 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55693b147400 con 0x55693b29c9b0 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f0059600700 10 client.25267 handle_command_reply: tid=0 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 client.25267 shutdown 2024-05-10T21:09:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 msgr2=0x55693b229a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f005c0a6d40 tx=0x7f005c00cf30 comp rx=0 tx=0).stop 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 shutdown_connections 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55693b29c9b0 0x55693b29edd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55693b21dbf0 0x55693b226a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55693b099e40 0x55693b229a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 --2- 172.21.15.50:0/3247777354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55693b217c50 0x55693b229f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f0059600700 0 client.25267 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 >> 172.21.15.50:0/3247777354 conn(0x55693b0d8a80 msgr2=0x55693b21b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 shutdown_connections 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 1 -- 172.21.15.50:0/3247777354 wait complete. 2024-05-10T21:09:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.349+0000 7f008143fb80 20 client.25267 trim_cache size 0 max 16384 2024-05-10T21:09:16.352 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:16.352 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:09:16.352 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:09:16.352 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:16.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 msgr2=0x7f00500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 0x7f00500b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55693b218a30 tx=0x7f0064004b40 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 msgr2=0x7f006c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f005c00eb30 tx=0x7f005c00ee40 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 shutdown_connections 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00500b2880 0x7f00500b4d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f006c109860 0x7f006c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f006c1045e0 0x7f006c076bd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 --2- 172.21.15.50:0/3469172317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f006c108f10 0x7f006c077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 >> 172.21.15.50:0/3469172317 conn(0x7f006c0ffed0 msgr2=0x7f006c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.350+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 shutdown_connections 2024-05-10T21:09:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:16.351+0000 7f0072800700 1 -- 172.21.15.50:0/3469172317 wait complete. 2024-05-10T21:09:16.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:16 smithi050 ceph-mon[29546]: pgmap v472: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:09:17.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:16 smithi092 ceph-mon[34415]: pgmap v472: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:09:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:16 smithi142 ceph-mon[34846]: pgmap v472: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T21:09:18.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 -- 172.21.15.50:0/4039443930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db810c5c0 msgr2=0x7f7db810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 --2- 172.21.15.50:0/4039443930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db810c5c0 0x7f7db810c9d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009990 tx=0x7f7da8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 -- 172.21.15.50:0/4039443930 shutdown_connections 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 --2- 172.21.15.50:0/4039443930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8106ed0 0x7f7db8113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 --2- 172.21.15.50:0/4039443930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db810c5c0 0x7f7db810c9d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 --2- 172.21.15.50:0/4039443930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db8106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 -- 172.21.15.50:0/4039443930 >> 172.21.15.50:0/4039443930 conn(0x7f7db8102080 msgr2=0x7f7db81044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 -- 172.21.15.50:0/4039443930 shutdown_connections 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.099+0000 7f7dbf000700 1 -- 172.21.15.50:0/4039443930 wait complete. 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.100+0000 7f7dbf000700 1 Processor -- start 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.100+0000 7f7dbf000700 1 -- start start 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 0x7f7db819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db810c5c0 0x7f7db81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8116740 con 0x7f7db8106ed0 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81165c0 con 0x7f7db8106580 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbf000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81168c0 con 0x7f7db810c5c0 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7db7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 0x7f7db819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7db7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 0x7f7db819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46296/0 (socket says 172.21.15.50:46296) 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7db7400700 1 -- 172.21.15.50:0/3552091938 learned_addr learned my addr 172.21.15.50:0/3552091938 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.101+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db810c5c0 0x7f7db81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 -- 172.21.15.50:0/3552091938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db810c5c0 msgr2=0x7f7db81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db810c5c0 0x7f7db81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 -- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 msgr2=0x7f7db819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 0x7f7db819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 -- 172.21.15.50:0/3552091938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db81a1f20 con 0x7f7db8106580 2024-05-10T21:09:18.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db819d050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7da80098e0 tx=0x7f7da80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db4c00700 1 -- 172.21.15.50:0/3552091938 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8027070 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db81a21b0 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db4c00700 1 -- 172.21.15.50:0/3552091938 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da8022470 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.102+0000 7f7db4c00700 1 -- 172.21.15.50:0/3552091938 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017650 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.103+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81a9b30 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.106+0000 7f7db4c00700 1 -- 172.21.15.50:0/3552091938 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7da802a070 con 0x7f7db8106580 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.107+0000 7f7dcdaabb80 1 Processor -- start 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 -- start start 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab4152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7db4c00700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 0x7f7d9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab40fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566ab3319d0 con 0x5566ab40f9a0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5566ab2cf710 con 0x5566ab4157e0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5566ab32d990 con 0x5566ab291df0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.108+0000 7f7db4c00700 1 -- 172.21.15.50:0/3552091938 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7da80e30e0 con 0x7f7db8106580 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab4152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 0x7f7d9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46290/0 (socket says 172.21.15.50:46290) 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 -- 172.21.15.50:0/3858817119 learned_addr learned my addr 172.21.15.50:0/3858817119 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab40fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 -- 172.21.15.50:0/3858817119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 msgr2=0x5566ab4152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab4152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 -- 172.21.15.50:0/3858817119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 msgr2=0x5566ab40fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab40fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 -- 172.21.15.50:0/3858817119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ab27e880 con 0x5566ab4157e0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab4152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.109+0000 7f7db7400700 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009d90 tx=0x7f7da4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e053d47b3dfa14cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.110+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 0x7f7d9c0b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009ec0 tx=0x7f7db0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.110+0000 7f7dad600700 1 -- 172.21.15.50:0/3858817119 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da401e070 con 0x5566ab4157e0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.110+0000 7f7dad600700 1 -- 172.21.15.50:0/3858817119 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da4019ae0 con 0x5566ab4157e0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.110+0000 7f7dad600700 1 -- 172.21.15.50:0/3858817119 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da400eb00 con 0x5566ab4157e0 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3858817119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 msgr2=0x5566ab40f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009d90 tx=0x7f7da4011720 comp rx=0 tx=0).stop 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3858817119 shutdown_connections 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab40fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab4152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3858817119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40f460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3858817119 >> 172.21.15.50:0/3858817119 conn(0x5566ab2d6f20 msgr2=0x5566ab40a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3858817119 shutdown_connections 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.111+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3858817119 wait complete. 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.112+0000 7f7dcdaabb80 1 -- start start 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.112+0000 7f7dcdaabb80 1 Processor -- start 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab419470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab4199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566ab41c2c0 con 0x5566ab40f9a0 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5566ab297ad0 con 0x5566ab4157e0 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5566ab41c440 con 0x5566ab291df0 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dcdaabb80 10 client.0 _ll_register_callbacks cb 0x5566ab2d8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:18.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab419470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dbd000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab4199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dbd000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab4199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46296/0 (socket says 172.21.15.50:46296) 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7dbd000700 1 -- 172.21.15.50:0/3015109392 learned_addr learned my addr 172.21.15.50:0/3015109392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52938/0 (socket says 172.21.15.50:52938) 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 -- 172.21.15.50:0/3015109392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 msgr2=0x5566ab4199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab4199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 -- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 msgr2=0x5566ab419470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab419470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7e00700 1 -- 172.21.15.50:0/3015109392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ab2f73d0 con 0x5566ab291df0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.113+0000 7f7db7400700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab419470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a0ebde65381423 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7da2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80ab890 con 0x5566ab291df0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7da80a6d50 con 0x5566ab291df0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da4019ae0 con 0x5566ab291df0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7da80aa530 con 0x5566ab291df0 2024-05-10T21:09:18.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7dcdaabb80 10 client.25279 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.114+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566ab41c320 con 0x5566ab291df0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da4019ae0 con 0x5566ab291df0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7da80b30a0 con 0x5566ab291df0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 10 client.25279 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 10 client.25279 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 20 client.25279 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab4296e0 0x5566ab42ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 4 client.25279 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ab3dc170 con 0x5566ab4296e0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.115+0000 7f7db7400700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab4296e0 0x5566ab42ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.116+0000 7f7db7400700 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab4296e0 0x5566ab42ab50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78084f25f2e9165a server_cookie=41dc38dabac188d0 in_seq=0 out_seq=0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.116+0000 7f7da2000700 10 client.25279 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7da2000700 1 -- 172.21.15.50:0/3015109392 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566ab3dc170 con 0x5566ab4296e0 2024-05-10T21:09:18.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7da2000700 10 client.25279 handle_command_reply: tid=0 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 client.25279 shutdown 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 msgr2=0x5566ab418f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da8005fd0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 shutdown_connections 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab4199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab4296e0 0x5566ab42ab50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab291df0 0x5566ab418f30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3015109392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab419470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 >> 172.21.15.50:0/3015109392 conn(0x5566ab2d6f20 msgr2=0x5566ab40fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7da2000700 0 client.25279 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 shutdown_connections 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.117+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3015109392 wait complete. 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.118+0000 7f7dcdaabb80 20 client.25279 trim_cache size 0 max 16384 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 Processor -- start 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 -- start start 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 0x5566ab40f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab4157e0 0x5566ab412c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566ab428e90 con 0x5566ab40f9a0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5566ab429560 con 0x5566ab291df0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.141+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5566ab429c30 con 0x5566ab4157e0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46314/0 (socket says 172.21.15.50:46314) 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7400700 1 -- 172.21.15.50:0/2280195578 learned_addr learned my addr 172.21.15.50:0/2280195578 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 0x5566ab40f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7dbd000700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab4157e0 0x5566ab412c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 -- 172.21.15.50:0/2280195578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab4157e0 msgr2=0x5566ab412c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab4157e0 0x5566ab412c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 -- 172.21.15.50:0/2280195578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 msgr2=0x5566ab428710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7e00700 1 -- 172.21.15.50:0/2280195578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ab410660 con 0x5566ab291df0 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.142+0000 7f7db7400700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.143+0000 7f7db7e00700 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 0x5566ab40f460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005f50 tx=0x7f7da80a6750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a3d9b48fc818c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.143+0000 7f7dacc00700 1 -- 172.21.15.50:0/2280195578 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80aae60 con 0x5566ab291df0 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.143+0000 7f7dacc00700 1 -- 172.21.15.50:0/2280195578 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da80aa270 con 0x5566ab291df0 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.143+0000 7f7dacc00700 1 -- 172.21.15.50:0/2280195578 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80aa760 con 0x5566ab291df0 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/2280195578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 msgr2=0x5566ab40f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 0x5566ab40f460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005f50 tx=0x7f7da80a6750 comp rx=0 tx=0).stop 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/2280195578 shutdown_connections 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab291df0 0x5566ab40f460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab4157e0 0x5566ab412c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/2280195578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab40f9a0 0x5566ab428710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/2280195578 >> 172.21.15.50:0/2280195578 conn(0x5566ab2d6f20 msgr2=0x5566ab428000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/2280195578 shutdown_connections 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/2280195578 wait complete. 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.144+0000 7f7dcdaabb80 1 -- start start 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 Processor -- start 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab291df0 0x5566ab434580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 0x5566ab40e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566ab42a9d0 con 0x5566ab291df0 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5566ab42a300 con 0x5566ab4157e0 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5566ab42b0a0 con 0x5566ab40f9a0 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.145+0000 7f7dcdaabb80 10 client.0 _ll_register_callbacks cb 0x5566ab2d8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:18.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab291df0 0x5566ab434580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7db7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 0x5566ab40e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7db7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 0x5566ab40e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52964/0 (socket says 172.21.15.50:52964) 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7db7400700 1 -- 172.21.15.50:0/3970156288 learned_addr learned my addr 172.21.15.50:0/3970156288 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 -- 172.21.15.50:0/3970156288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 msgr2=0x5566ab40e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 0x5566ab40e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 -- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab291df0 msgr2=0x5566ab434580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:18.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab291df0 0x5566ab434580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 -- 172.21.15.50:0/3970156288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ab40f1e0 con 0x5566ab4157e0 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7dbd000700 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40ea10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7db0018c20 tx=0x7f7db0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e0b0985751d12c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.146+0000 7f7da2000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db002b030 con 0x5566ab4157e0 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da80aa270 con 0x5566ab4157e0 2024-05-10T21:09:18.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7db0012040 con 0x5566ab4157e0 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7db0037050 con 0x5566ab4157e0 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7dcdaabb80 10 client.25292 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.147+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566ab494c00 con 0x5566ab4157e0 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da80aa270 con 0x5566ab4157e0 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7db0004020 con 0x5566ab4157e0 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 10 client.25292 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:18.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 10 client.25292 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 20 client.25292 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab497050 0x5566ab4984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 4 client.25292 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.148+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "status"}) -- 0x5566ab33f3b0 con 0x5566ab497050 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.149+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab497050 0x5566ab4984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.149+0000 7f7db7e00700 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab497050 0x5566ab4984c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf576ebe2cf4a217 server_cookie=f5c59260a5077fa1 in_seq=0 out_seq=0 2024-05-10T21:09:18.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.149+0000 7f7da2000700 10 client.25292 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7da2000700 1 -- 172.21.15.50:0/3970156288 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5566ab33f3b0 con 0x5566ab497050 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7da2000700 10 client.25292 handle_command_reply: tid=0 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 client.25292 shutdown 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 msgr2=0x5566ab40ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40ea10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7db0018c20 tx=0x7f7db0004b40 comp rx=0 tx=0).stop 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 shutdown_connections 2024-05-10T21:09:18.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5566ab4157e0 0x5566ab40ea10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5566ab497050 0x5566ab4984c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566ab291df0 0x5566ab434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 --2- 172.21.15.50:0/3970156288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566ab40f9a0 0x5566ab40e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7da2000700 0 client.25292 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 >> 172.21.15.50:0/3970156288 conn(0x5566ab2d6f20 msgr2=0x5566ab404050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 shutdown_connections 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.150+0000 7f7dcdaabb80 1 -- 172.21.15.50:0/3970156288 wait complete. 2024-05-10T21:09:18.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dcdaabb80 20 client.25292 trim_cache size 0 max 16384 2024-05-10T21:09:18.172 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:09:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24520, 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 275.35832165099998, 2024-05-10T21:09:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-05-10T21:09:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:09:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 82, 2024-05-10T21:09:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 316.30866125199998 2024-05-10T21:09:18.174 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 msgr2=0x7f7d9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 0x7f7d9c0b4d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009ec0 tx=0x7f7db0009920 comp rx=0 tx=0).stop 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 msgr2=0x7f7db819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db819d050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7da80098e0 tx=0x7f7da80057c0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 shutdown_connections 2024-05-10T21:09:18.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7d9c0b2880 0x7f7d9c0b4d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7db8106580 0x7f7db819d050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.151+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db810c5c0 0x7f7db81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.152+0000 7f7dbf000700 1 --2- 172.21.15.50:0/3552091938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7db8106ed0 0x7f7db819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.152+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 >> 172.21.15.50:0/3552091938 conn(0x7f7db8102080 msgr2=0x7f7db8104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.152+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 shutdown_connections 2024-05-10T21:09:18.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:18.152+0000 7f7dbf000700 1 -- 172.21.15.50:0/3552091938 wait complete. 2024-05-10T21:09:18.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:09:18.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:09:19.157 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:18 smithi050 ceph-mon[29546]: pgmap v473: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:09:19.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T21:09:19.356 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:19.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:18 smithi092 ceph-mon[34415]: pgmap v473: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:09:19.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:18 smithi142 ceph-mon[34846]: pgmap v473: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-10T21:09:20.769 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:21.159 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:20 smithi050 ceph-mon[29546]: pgmap v474: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:21.170 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:20 smithi092 ceph-mon[34415]: pgmap v474: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:20 smithi142 ceph-mon[34846]: pgmap v474: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-10T21:09:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 -- 172.21.15.50:0/3867493158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810b870 msgr2=0x7f7f981131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 --2- 172.21.15.50:0/3867493158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810b870 0x7f7f981131d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009fc0 tx=0x7f7f9000b330 comp rx=0 tx=0).stop 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 -- 172.21.15.50:0/3867493158 shutdown_connections 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 --2- 172.21.15.50:0/3867493158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f98106640 0x7f7f98106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 --2- 172.21.15.50:0/3867493158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810b870 0x7f7f981131d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 --2- 172.21.15.50:0/3867493158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f9810af20 0x7f7f9810b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 -- 172.21.15.50:0/3867493158 >> 172.21.15.50:0/3867493158 conn(0x7f7f98102030 msgr2=0x7f7f98104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 -- 172.21.15.50:0/3867493158 shutdown_connections 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.272+0000 7f7f97200700 1 -- 172.21.15.50:0/3867493158 wait complete. 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.273+0000 7f7f97200700 1 Processor -- start 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.273+0000 7f7f97200700 1 -- start start 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 0x7f7f9819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 0x7f7f981a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981164f0 con 0x7f7f98106640 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98116370 con 0x7f7f9810b870 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f97200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98116670 con 0x7f7f9810af20 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f95e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f95e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46342/0 (socket says 172.21.15.50:46342) 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f95e00700 1 -- 172.21.15.50:0/963016057 learned_addr learned my addr 172.21.15.50:0/963016057 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f8fe00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 0x7f7f9819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f96800700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 0x7f7f981a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f95e00700 1 -- 172.21.15.50:0/963016057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 msgr2=0x7f7f9819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.274+0000 7f7f95e00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 0x7f7f9819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f95e00700 1 -- 172.21.15.50:0/963016057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 msgr2=0x7f7f981a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f95e00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 0x7f7f981a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f95e00700 1 -- 172.21.15.50:0/963016057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f981a1cb0 con 0x7f7f98106640 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f96800700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 0x7f7f981a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:22.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f8fe00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 0x7f7f9819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f95e00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9a0 tx=0x7f7f8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800c9c0 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8800cb60 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88018680 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f981a1fa0 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.275+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f981a98c0 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.277+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f980512c0 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.279+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f88010430 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.282+0000 7f7f8ea00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 0x7f7f7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.282+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f7f880e27c0 con 0x7f7f98106640 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.282+0000 7f7f8fe00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 0x7f7f7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.283+0000 7f7f8fe00700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 0x7f7f7c0b4e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f98105af0 tx=0x7f7f80009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:22.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.285+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f880aa4b0 con 0x7f7f98106640 2024-05-10T21:09:22.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.593+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f9802e5f0 con 0x7f7f98106640 2024-05-10T21:09:22.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.594+0000 7f7f8ea00700 1 -- 172.21.15.50:0/963016057 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f7f880a9db0 con 0x7f7f98106640 2024-05-10T21:09:22.594 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:09:22.594 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:09:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 msgr2=0x7f7f7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 0x7f7f7c0b4e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f98105af0 tx=0x7f7f80009310 comp rx=0 tx=0).stop 2024-05-10T21:09:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 msgr2=0x7f7f9819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9a0 tx=0x7f7f8800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 shutdown_connections 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f7c0b2900 0x7f7f7c0b4e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f9810b870 0x7f7f981a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f98106640 0x7f7f9819ce80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 --2- 172.21.15.50:0/963016057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f9810af20 0x7f7f9819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 >> 172.21.15.50:0/963016057 conn(0x7f7f98102030 msgr2=0x7f7f98109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.597+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 shutdown_connections 2024-05-10T21:09:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:22.598+0000 7f7f97200700 1 -- 172.21.15.50:0/963016057 wait complete. 2024-05-10T21:09:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:09:23.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:22 smithi050 ceph-mon[29546]: pgmap v475: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:23.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:22 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/963016057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:23.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:22 smithi092 ceph-mon[34415]: pgmap v475: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:23.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:22 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/963016057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:22 smithi142 ceph-mon[34846]: pgmap v475: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:22 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/963016057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.569+0000 7fb4a5a00700 1 -- 172.21.15.50:0/2125107930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0076680 msgr2=0x7fb4a0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.569+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/2125107930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0076680 0x7fb4a0110470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 -- 172.21.15.50:0/2125107930 shutdown_connections 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/2125107930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a01109b0 0x7fb4a0112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/2125107930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a0075d30 0x7fb4a0076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/2125107930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0076680 0x7fb4a0110470 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 -- 172.21.15.50:0/2125107930 >> 172.21.15.50:0/2125107930 conn(0x7fb4a0079d10 msgr2=0x7fb4a007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 -- 172.21.15.50:0/2125107930 shutdown_connections 2024-05-10T21:09:23.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.570+0000 7fb4a5a00700 1 -- 172.21.15.50:0/2125107930 wait complete. 2024-05-10T21:09:23.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.571+0000 7fb4a5a00700 1 Processor -- start 2024-05-10T21:09:23.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 -- start start 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 0x7fb4a019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 0x7fb4a019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a01109b0 0x7fb4a01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0116500 con 0x7fb4a01109b0 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0116380 con 0x7fb4a0076680 2024-05-10T21:09:23.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.572+0000 7fb4a5a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0116680 con 0x7fb4a0075d30 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 0x7fb4a019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 0x7fb4a019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52980/0 (socket says 172.21.15.50:52980) 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb49fe00700 1 -- 172.21.15.50:0/1220620285 learned_addr learned my addr 172.21.15.50:0/1220620285 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 0x7fb4a019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb4a5000700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a01109b0 0x7fb4a01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 -- 172.21.15.50:0/1220620285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 msgr2=0x7fb4a019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 0x7fb4a019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 -- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a01109b0 msgr2=0x7fb4a01a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a01109b0 0x7fb4a01a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.573+0000 7fb497e00700 1 -- 172.21.15.50:0/1220620285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01a1e20 con 0x7fb4a0076680 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb497e00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 0x7fb4a019d4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098e0 tx=0x7fb4880057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb49d600700 1 -- 172.21.15.50:0/1220620285 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488027070 con 0x7fb4a0076680 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a01a20b0 con 0x7fb4a0076680 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb49d600700 1 -- 172.21.15.50:0/1220620285 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb488022470 con 0x7fb4a0076680 2024-05-10T21:09:23.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb49d600700 1 -- 172.21.15.50:0/1220620285 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488017650 con 0x7fb4a0076680 2024-05-10T21:09:23.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.574+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a2590 con 0x7fb4a0076680 2024-05-10T21:09:23.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.578+0000 7fb49d600700 1 -- 172.21.15.50:0/1220620285 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb48802a070 con 0x7fb4a0076680 2024-05-10T21:09:23.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.580+0000 7fb49d600700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 0x7fb4840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.580+0000 7fb49d600700 1 -- 172.21.15.50:0/1220620285 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb4880e3090 con 0x7fb4a0076680 2024-05-10T21:09:23.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.580+0000 7fb49fe00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 0x7fb4840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.581+0000 7fb49fe00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 0x7fb4840b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb490009d20 tx=0x7fb490009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.581+0000 7fb4b5b11b80 1 Processor -- start 2024-05-10T21:09:23.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.581+0000 7fb4b5b11b80 1 -- start start 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 0x55c23cd896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd8a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c23ccab9d0 con 0x55c23cd8fa30 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c23cc49710 con 0x55c23cd89bf0 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c23cca7990 con 0x55c23cc0bdf0 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb497e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 0x55c23cd896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52996/0 (socket says 172.21.15.50:52996) 2024-05-10T21:09:23.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb49fe00700 1 -- 172.21.15.50:0/3327669343 learned_addr learned my addr 172.21.15.50:0/3327669343 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.582+0000 7fb4a5000700 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd8a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 -- 172.21.15.50:0/3327669343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 msgr2=0x55c23cd8a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd8a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 -- 172.21.15.50:0/3327669343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 msgr2=0x55c23cd896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 0x55c23cd896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 -- 172.21.15.50:0/3327669343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23cbf8880 con 0x55c23cc0bdf0 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb497e00700 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 0x55c23cd896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb49fe00700 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e540 tx=0x7fb490014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f539625c3fef91ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.583+0000 7fb495600700 1 -- 172.21.15.50:0/3327669343 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490026430 con 0x55c23cc0bdf0 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.584+0000 7fb495600700 1 -- 172.21.15.50:0/3327669343 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49001c8c0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.584+0000 7fb495600700 1 -- 172.21.15.50:0/3327669343 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49001d850 con 0x55c23cc0bdf0 2024-05-10T21:09:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.584+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/3327669343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 msgr2=0x55c23cd8f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.584+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e540 tx=0x7fb490014000 comp rx=0 tx=0).stop 2024-05-10T21:09:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/3327669343 shutdown_connections 2024-05-10T21:09:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd8a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cc0bdf0 0x55c23cd8f4f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/3327669343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd8fa30 0x55c23cd896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/3327669343 >> 172.21.15.50:0/3327669343 conn(0x55c23cc50f20 msgr2=0x55c23cd843b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/3327669343 shutdown_connections 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/3327669343 wait complete. 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.585+0000 7fb4b5b11b80 1 -- start start 2024-05-10T21:09:23.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.586+0000 7fb4b5b11b80 1 Processor -- start 2024-05-10T21:09:23.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd89bf0 0x55c23cd93620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cd93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c23cd96440 con 0x55c23cd89bf0 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c23cc11ad0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c23cd965c0 con 0x55c23cd8fa30 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb4b5b11b80 10 client.0 _ll_register_callbacks cb 0x55c23cc52c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46392/0 (socket says 172.21.15.50:46392) 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.587+0000 7fb49fe00700 1 -- 172.21.15.50:0/1164832948 learned_addr learned my addr 172.21.15.50:0/1164832948 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 -- 172.21.15.50:0/1164832948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 msgr2=0x55c23cd93b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cd93b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 -- 172.21.15.50:0/1164832948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd89bf0 msgr2=0x55c23cd93620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd89bf0 0x55c23cd93620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 -- 172.21.15.50:0/1164832948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23cc713d0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb49fe00700 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb49001c410 tx=0x7fb49001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee99ffe3c6796aca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb48e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.588+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490034060 con 0x55c23cc0bdf0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb49001cc80 con 0x55c23cc0bdf0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49001c8c0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb490005040 con 0x55c23cc0bdf0 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb4b5b11b80 10 client.25310 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:23.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c23cda0f80 con 0x55c23cc0bdf0 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49001c8c0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.589+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb490004bc0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 10 client.25310 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 10 client.25310 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 20 client.25310 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cda3250 0x55c23cda46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 4 client.25310 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c23cd56170 con 0x55c23cda3250 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.590+0000 7fb497e00700 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cda3250 0x55c23cda46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.591+0000 7fb497e00700 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cda3250 0x55c23cda46c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15db6c3019ca2df2 server_cookie=9b5b7d7339a6c8ca in_seq=0 out_seq=0 2024-05-10T21:09:23.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.591+0000 7fb48e000700 10 client.25310 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:23.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.591+0000 7fb48e000700 1 -- 172.21.15.50:0/1164832948 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c23cd56170 con 0x55c23cda3250 2024-05-10T21:09:23.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.591+0000 7fb48e000700 10 client.25310 handle_command_reply: tid=0 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.591+0000 7fb4b5b11b80 1 client.25310 shutdown 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 msgr2=0x55c23cd930e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb49001c410 tx=0x7fb49001c4f0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 shutdown_connections 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cda3250 0x55c23cda46c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cc0bdf0 0x55c23cd930e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cd93b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/1164832948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cd89bf0 0x55c23cd93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb48e000700 0 client.25310 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 >> 172.21.15.50:0/1164832948 conn(0x55c23cc50f20 msgr2=0x55c23cd886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 shutdown_connections 2024-05-10T21:09:23.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/1164832948 wait complete. 2024-05-10T21:09:23.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.592+0000 7fb4b5b11b80 20 client.25310 trim_cache size 0 max 16384 2024-05-10T21:09:23.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 Processor -- start 2024-05-10T21:09:23.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 -- start start 2024-05-10T21:09:23.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 0x55c23cd7e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 0x55c23cd9f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda2a00 con 0x55c23cc0bdf0 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda30d0 con 0x55c23cd8fa30 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.616+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda37a0 con 0x55c23cd89bf0 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46380/0 (socket says 172.21.15.50:46380) 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb49fe00700 1 -- 172.21.15.50:0/2778611150 learned_addr learned my addr 172.21.15.50:0/2778611150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb4a5000700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 0x55c23cd9f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 0x55c23cd7e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 -- 172.21.15.50:0/2778611150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 msgr2=0x55c23cd9f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 0x55c23cd9f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 -- 172.21.15.50:0/2778611150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 msgr2=0x55c23cd896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb497e00700 1 -- 172.21.15.50:0/2778611150 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23cd80bf0 con 0x55c23cd89bf0 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb49fe00700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.617+0000 7fb4a5000700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 0x55c23cd9f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.618+0000 7fb497e00700 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 0x55c23cd7e5c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e0bc0 tx=0x7fb4880e0a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4979fe312a57df21 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.618+0000 7fb494c00700 1 -- 172.21.15.50:0/2778611150 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4880aa770 con 0x55c23cd89bf0 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.618+0000 7fb494c00700 1 -- 172.21.15.50:0/2778611150 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4880aeb20 con 0x55c23cd89bf0 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.618+0000 7fb494c00700 1 -- 172.21.15.50:0/2778611150 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4880aa9d0 con 0x55c23cd89bf0 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2778611150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 msgr2=0x55c23cd7e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 0x55c23cd7e5c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e0bc0 tx=0x7fb4880e0a90 comp rx=0 tx=0).stop 2024-05-10T21:09:23.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2778611150 shutdown_connections 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd8fa30 0x55c23cd9f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2778611150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd89bf0 0x55c23cd7e5c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2778611150 >> 172.21.15.50:0/2778611150 conn(0x55c23cc50f20 msgr2=0x55c23cd8e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2778611150 shutdown_connections 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.619+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2778611150 wait complete. 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.620+0000 7fb4b5b11b80 1 -- start start 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.620+0000 7fb4b5b11b80 1 Processor -- start 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.620+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.620+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.620+0000 7fb4b5b11b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cdacb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda4540 con 0x55c23cc0bdf0 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda3e70 con 0x55c23cd89bf0 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb4b5b11b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c23cda4c10 con 0x55c23cd8fa30 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb4b5b11b80 10 client.0 _ll_register_callbacks cb 0x55c23cc52c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb497e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb497e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46398/0 (socket says 172.21.15.50:46398) 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb497e00700 1 -- 172.21.15.50:0/2016064654 learned_addr learned my addr 172.21.15.50:0/2016064654 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:23.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb4a5000700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cdacb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 -- 172.21.15.50:0/2016064654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 msgr2=0x55c23cdacb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cdacb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 -- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 msgr2=0x55c23cd930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb49fe00700 1 -- 172.21.15.50:0/2016064654 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23cd852d0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.621+0000 7fb497e00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb49fe00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd92b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb490009be0 tx=0x7fb49001cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60320fe36cc8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb48e000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49003f070 con 0x55c23cc0bdf0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb49001c820 con 0x55c23cc0bdf0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4880aeb20 con 0x55c23cc0bdf0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb490034040 con 0x55c23cc0bdf0 2024-05-10T21:09:23.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb4b5b11b80 10 client.15198 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.622+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c23cdae020 con 0x55c23cc0bdf0 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4880aeb20 con 0x55c23cc0bdf0 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb4900265a0 con 0x55c23cc0bdf0 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 20 client.15198 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cdae310 0x55c23ce11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 4 client.15198 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T21:09:23.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.623+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "status"}) -- 0x55c23ccb93b0 con 0x55c23cdae310 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb497e00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cdae310 0x55c23ce11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb497e00700 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cdae310 0x55c23ce11820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30ea4bca08ac86f server_cookie=ae4dda8bb9594ece in_seq=0 out_seq=0 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb48e000700 10 client.15198 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb48e000700 1 -- 172.21.15.50:0/2016064654 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c23ccb93b0 con 0x55c23cdae310 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb48e000700 10 client.15198 handle_command_reply: tid=0 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb4b5b11b80 1 client.15198 shutdown 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 msgr2=0x55c23cd92b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.624+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd92b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb490009be0 tx=0x7fb49001cd00 comp rx=0 tx=0).stop 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 shutdown_connections 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c23cdae310 0x55c23ce11820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c23cd8fa30 0x55c23cdacb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c23cc0bdf0 0x55c23cd92b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 --2- 172.21.15.50:0/2016064654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c23cd89bf0 0x55c23cd930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 >> 172.21.15.50:0/2016064654 conn(0x55c23cc50f20 msgr2=0x55c23cd8ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb48e000700 0 client.15198 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 shutdown_connections 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 1 -- 172.21.15.50:0/2016064654 wait complete. 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.625+0000 7fb4b5b11b80 20 client.15198 trim_cache size 0 max 16384 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:23.630 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "55ba4db6-0f0f-11ef-bc98-c7b262605968", 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24539, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 296.68143388099998, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 82, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 82, 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 319.26217964300002 2024-05-10T21:09:23.631 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 msgr2=0x7fb4840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 0x7fb4840b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb490009d20 tx=0x7fb490009260 comp rx=0 tx=0).stop 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 msgr2=0x7fb4a019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 0x7fb4a019d4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098e0 tx=0x7fb4880057c0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 shutdown_connections 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4840b2880 0x7fb4840b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4a01109b0 0x7fb4a01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a0075d30 0x7fb4a019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 --2- 172.21.15.50:0/1220620285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a0076680 0x7fb4a019d4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:23.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 >> 172.21.15.50:0/1220620285 conn(0x7fb4a0079d10 msgr2=0x7fb4a0102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:23.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 shutdown_connections 2024-05-10T21:09:23.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:23.626+0000 7fb4a5a00700 1 -- 172.21.15.50:0/1220620285 wait complete. 2024-05-10T21:09:23.723 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:09:24.550 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-10T21:09:24.612 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:25.101 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:24 smithi050 ceph-mon[29546]: pgmap v476: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-10T21:09:25.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:24 smithi092 ceph-mon[34415]: pgmap v476: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-10T21:09:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:24 smithi142 ceph-mon[34846]: pgmap v476: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 -- 172.21.15.50:0/989989606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0077790 msgr2=0x7fadd0110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 --2- 172.21.15.50:0/989989606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0077790 0x7fadd0110660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009990 tx=0x7fadbc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 -- 172.21.15.50:0/989989606 shutdown_connections 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 --2- 172.21.15.50:0/989989606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 0x7fadd0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 --2- 172.21.15.50:0/989989606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0076e40 0x7fadd0077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 --2- 172.21.15.50:0/989989606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0077790 0x7fadd0110660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 -- 172.21.15.50:0/989989606 >> 172.21.15.50:0/989989606 conn(0x7fadd00ffec0 msgr2=0x7fadd0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 -- 172.21.15.50:0/989989606 shutdown_connections 2024-05-10T21:09:26.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.802+0000 7fadd8000700 1 -- 172.21.15.50:0/989989606 wait complete. 2024-05-10T21:09:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.803+0000 7fadd8000700 1 Processor -- start 2024-05-10T21:09:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.803+0000 7fadd8000700 1 -- start start 2024-05-10T21:09:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 0x7fadd0104800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 0x7fadd0104d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01166f0 con 0x7fadd0076e40 2024-05-10T21:09:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0116570 con 0x7fadd0110ba0 2024-05-10T21:09:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd8000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0116870 con 0x7fadd0077790 2024-05-10T21:09:26.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd5600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd5600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59842/0 (socket says 172.21.15.50:59842) 2024-05-10T21:09:26.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd5600700 1 -- 172.21.15.50:0/3506603478 learned_addr learned my addr 172.21.15.50:0/3506603478 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:26.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.804+0000 7fadd4c00700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 0x7fadd0104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd6000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 0x7fadd0104d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 -- 172.21.15.50:0/3506603478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 msgr2=0x7fadd0104d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 0x7fadd0104d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 -- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 msgr2=0x7fadd01042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 -- 172.21.15.50:0/3506603478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01ad6a0 con 0x7fadd0077790 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd5600700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.805+0000 7fadd4c00700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 0x7fadd0104800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098e0 tx=0x7fadbc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.806+0000 7fadce000700 1 -- 172.21.15.50:0/3506603478 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc027070 con 0x7fadd0077790 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.806+0000 7fadce000700 1 -- 172.21.15.50:0/3506603478 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadbc022470 con 0x7fadd0077790 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.806+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01ad8d0 con 0x7fadd0077790 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.806+0000 7fadce000700 1 -- 172.21.15.50:0/3506603478 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc017650 con 0x7fadd0077790 2024-05-10T21:09:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.806+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01addb0 con 0x7fadd0077790 2024-05-10T21:09:26.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.810+0000 7fadce000700 1 -- 172.21.15.50:0/3506603478 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadbc02a070 con 0x7fadd0077790 2024-05-10T21:09:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 Processor -- start 2024-05-10T21:09:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 -- start start 2024-05-10T21:09:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbf12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbf1830 0x564a2cbeb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a2cb0da20 con 0x564a2ca6de40 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564a2caab760 con 0x564a2cbf1830 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564a2cb099e0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.811+0000 7fadce000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 0x7fadb80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47286/0 (socket says 172.21.15.50:47286) 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 -- 172.21.15.50:0/748806156 learned_addr learned my addr 172.21.15.50:0/748806156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd5600700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbf1830 0x564a2cbeb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd6000700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbf12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadce000700 1 -- 172.21.15.50:0/3506603478 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fadbc0e2fe0 con 0x7fadd0077790 2024-05-10T21:09:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd6000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 0x7fadb80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 -- 172.21.15.50:0/748806156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbf1830 msgr2=0x564a2cbeb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbf1830 0x564a2cbeb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 -- 172.21.15.50:0/748806156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 msgr2=0x564a2cbf12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbf12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd4c00700 1 -- 172.21.15.50:0/748806156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2ca5a8d0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.812+0000 7fadd6000700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbf12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.813+0000 7fadd4c00700 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0a9e90 tx=0x7fadbc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c9bd3080222e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.813+0000 7fadd6000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 0x7fadb80b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fadd0078ce0 tx=0x7fadc800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.813+0000 7fadc2a00700 1 -- 172.21.15.50:0/748806156 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0aada0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.813+0000 7fadc2a00700 1 -- 172.21.15.50:0/748806156 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadbc0b3470 con 0x564a2cbeb9f0 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.813+0000 7fadc2a00700 1 -- 172.21.15.50:0/748806156 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc022920 con 0x564a2cbeb9f0 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 -- 172.21.15.50:0/748806156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 msgr2=0x564a2cbebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0a9e90 tx=0x7fadbc005fd0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 -- 172.21.15.50:0/748806156 shutdown_connections 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbeb9f0 0x564a2cbebe00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbf12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/748806156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbf1830 0x564a2cbeb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.814+0000 7fade6ba9b80 1 -- 172.21.15.50:0/748806156 >> 172.21.15.50:0/748806156 conn(0x564a2cab2f70 msgr2=0x564a2cbe62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.815+0000 7fade6ba9b80 1 -- 172.21.15.50:0/748806156 shutdown_connections 2024-05-10T21:09:26.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.815+0000 7fade6ba9b80 1 -- 172.21.15.50:0/748806156 wait complete. 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.815+0000 7fade6ba9b80 1 -- start start 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.815+0000 7fade6ba9b80 1 Processor -- start 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a2cbf80d0 con 0x564a2cbf1830 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564a2ca73b20 con 0x564a2cbeb9f0 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564a2cbf8250 con 0x564a2ca6de40 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fade6ba9b80 10 client.0 _ll_register_callbacks cb 0x564a2cab4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.816+0000 7fadd5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47292/0 (socket says 172.21.15.50:47292) 2024-05-10T21:09:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd6000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 -- 172.21.15.50:0/2334166075 learned_addr learned my addr 172.21.15.50:0/2334166075 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd4c00700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 -- 172.21.15.50:0/2334166075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 msgr2=0x564a2cbf54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 -- 172.21.15.50:0/2334166075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 msgr2=0x564a2cbf5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 -- 172.21.15.50:0/2334166075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2cad3420 con 0x564a2ca6de40 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadd5600700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc4011d10 tx=0x7fadc400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5364733001fc3803 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.817+0000 7fadb7400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc400fd90 con 0x564a2ca6de40 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fadc4017070 con 0x564a2ca6de40 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadbc0b3470 con 0x564a2ca6de40 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadc4004050 con 0x564a2ca6de40 2024-05-10T21:09:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fade6ba9b80 10 client.25318 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a2cc02b10 con 0x564a2ca6de40 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.818+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadbc0b3470 con 0x564a2ca6de40 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fadc40050b0 con 0x564a2ca6de40 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 10 client.25318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:09:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 20 client.25318 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc050a0 0x564a2cc07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 4 client.25318 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2cbb81c0 con 0x564a2cc050a0 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.819+0000 7fadd4c00700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc050a0 0x564a2cc07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.820+0000 7fadd4c00700 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc050a0 0x564a2cc07570 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49549bd1fe32d285 server_cookie=8241ffb9ecf22946 in_seq=0 out_seq=0 2024-05-10T21:09:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.820+0000 7fadb7400700 10 client.25318 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fadb7400700 1 -- 172.21.15.50:0/2334166075 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a2cbb81c0 con 0x564a2cc050a0 2024-05-10T21:09:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fadb7400700 10 client.25318 handle_command_reply: tid=0 2024-05-10T21:09:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 client.25318 shutdown 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 msgr2=0x564a2cbf4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc4011d10 tx=0x7fadc400f6a0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 shutdown_connections 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc050a0 0x564a2cc07570 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbf4f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/2334166075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fadb7400700 0 client.25318 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 >> 172.21.15.50:0/2334166075 conn(0x564a2cab2f70 msgr2=0x564a2cbea4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 shutdown_connections 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 1 -- 172.21.15.50:0/2334166075 wait complete. 2024-05-10T21:09:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.821+0000 7fade6ba9b80 20 client.25318 trim_cache size 0 max 16384 2024-05-10T21:09:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.838+0000 7fade6ba9b80 1 Processor -- start 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.838+0000 7fade6ba9b80 1 -- start start 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b7a70 con 0x564a2cbf1830 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b5300 con 0x564a2cbeb9f0 2024-05-10T21:09:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b3170 con 0x564a2ca6de40 2024-05-10T21:09:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fadd5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fadd5600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47308/0 (socket says 172.21.15.50:47308) 2024-05-10T21:09:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fadd5600700 1 -- 172.21.15.50:0/3534501350 learned_addr learned my addr 172.21.15.50:0/3534501350 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fadd6000700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.839+0000 7fadd4c00700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 -- 172.21.15.50:0/3534501350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 msgr2=0x564a2cbf42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 -- 172.21.15.50:0/3534501350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 msgr2=0x564a2cbf5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 -- 172.21.15.50:0/3534501350 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2cbe6ff0 con 0x564a2ca6de40 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd6000700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadd5600700 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadc40054a0 tx=0x7fadc4005260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160dc339eae1750a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadc2000700 1 -- 172.21.15.50:0/3534501350 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc400fd90 con 0x564a2ca6de40 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadc2000700 1 -- 172.21.15.50:0/3534501350 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc4003b30 con 0x564a2ca6de40 2024-05-10T21:09:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.840+0000 7fadc2000700 1 -- 172.21.15.50:0/3534501350 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4026930 con 0x564a2ca6de40 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 -- 172.21.15.50:0/3534501350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 msgr2=0x564a2cbeb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadc40054a0 tx=0x7fadc4005260 comp rx=0 tx=0).stop 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 -- 172.21.15.50:0/3534501350 shutdown_connections 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2cbf1830 0x564a2cbf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2ca6de40 0x564a2cbeb4b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.841+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/3534501350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbf42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.842+0000 7fade6ba9b80 1 -- 172.21.15.50:0/3534501350 >> 172.21.15.50:0/3534501350 conn(0x564a2cab2f70 msgr2=0x564a2cbf0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.842+0000 7fade6ba9b80 1 -- 172.21.15.50:0/3534501350 shutdown_connections 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.842+0000 7fade6ba9b80 1 -- 172.21.15.50:0/3534501350 wait complete. 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.842+0000 7fade6ba9b80 1 -- start start 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 Processor -- start 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbfdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 0x564a2cc0bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b37c0 con 0x564a2ca6de40 2024-05-10T21:09:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b4c30 con 0x564a2cbeb9f0 2024-05-10T21:09:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd01b3e90 con 0x564a2cbf1830 2024-05-10T21:09:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.843+0000 7fade6ba9b80 10 client.0 _ll_register_callbacks cb 0x564a2cab4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd5600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 0x564a2cc0bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd5600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59866/0 (socket says 172.21.15.50:59866) 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbfdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 0x564a2cc0bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47324/0 (socket says 172.21.15.50:47324) 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd5600700 1 -- 172.21.15.50:0/211986574 learned_addr learned my addr 172.21.15.50:0/211986574 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 -- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 msgr2=0x564a2cc0bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 0x564a2cc0bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 -- 172.21.15.50:0/211986574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 msgr2=0x564a2cbfd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 -- 172.21.15.50:0/211986574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2cc0c6d0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd5600700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadd4c00700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbfdaf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0aa630 tx=0x7fadbc005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dccdee15346217e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.844+0000 7fadb7400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b3470 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc4003b30 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fadbc022920 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadbc0aada0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fade6ba9b80 10 client.25334 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a2cc6f3b0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.845+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc4003b30 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fadbc0afbe0 con 0x564a2cbeb9f0 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 10 client.25334 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 10 client.25334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 20 client.25334 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc6f910 0x564a2cc71d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 4 client.25334 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:09:26.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a2cb1b400 con 0x564a2cc6f910 2024-05-10T21:09:26.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.846+0000 7fadd5600700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc6f910 0x564a2cc71d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fadd5600700 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc6f910 0x564a2cc71d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6d1f806caa11656 server_cookie=7fe6a0d5fcf9cb4c in_seq=0 out_seq=0 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fadb7400700 10 client.25334 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fadb7400700 1 -- 172.21.15.50:0/211986574 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a2cb1b400 con 0x564a2cc6f910 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fadb7400700 10 client.25334 handle_command_reply: tid=0 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 client.25334 shutdown 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 msgr2=0x564a2cbfdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbfdaf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0aa630 tx=0x7fadbc005040 comp rx=0 tx=0).stop 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 shutdown_connections 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x564a2cc6f910 0x564a2cc71d30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564a2cbf1830 0x564a2cc0bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a2ca6de40 0x564a2cbfd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 --2- 172.21.15.50:0/211986574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a2cbeb9f0 0x564a2cbfdaf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fadb7400700 0 client.25334 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.847+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 >> 172.21.15.50:0/211986574 conn(0x564a2cab2f70 msgr2=0x564a2cbeecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 shutdown_connections 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fade6ba9b80 1 -- 172.21.15.50:0/211986574 wait complete. 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fade6ba9b80 20 client.25334 trim_cache size 0 max 16384 2024-05-10T21:09:26.850 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 msgr2=0x7fadb80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 0x7fadb80b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fadd0078ce0 tx=0x7fadc800b040 comp rx=0 tx=0).stop 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 msgr2=0x7fadd0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.848+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 0x7fadd0104800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098e0 tx=0x7fadbc0057c0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 shutdown_connections 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fadb80b2880 0x7fadb80b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadd0110ba0 0x7fadd0104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd0076e40 0x7fadd01042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 --2- 172.21.15.50:0/3506603478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd0077790 0x7fadd0104800 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:26.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 >> 172.21.15.50:0/3506603478 conn(0x7fadd00ffec0 msgr2=0x7fadd01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:26.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 shutdown_connections 2024-05-10T21:09:26.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:26.849+0000 7fadd8000700 1 -- 172.21.15.50:0/3506603478 wait complete. 2024-05-10T21:09:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 -- 172.21.15.50:0/1119657361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60103c70 msgr2=0x7efe60104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 --2- 172.21.15.50:0/1119657361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60103c70 0x7efe60104970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 -- 172.21.15.50:0/1119657361 shutdown_connections 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 --2- 172.21.15.50:0/1119657361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 0x7efe6010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 --2- 172.21.15.50:0/1119657361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60104eb0 0x7efe601052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.126+0000 7efe67600640 1 --2- 172.21.15.50:0/1119657361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60103c70 0x7efe60104970 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.127+0000 7efe67600640 1 -- 172.21.15.50:0/1119657361 >> 172.21.15.50:0/1119657361 conn(0x7efe600fee60 msgr2=0x7efe60101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.127+0000 7efe67600640 1 -- 172.21.15.50:0/1119657361 shutdown_connections 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.127+0000 7efe67600640 1 -- 172.21.15.50:0/1119657361 wait complete. 2024-05-10T21:09:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.128+0000 7efe67600640 1 Processor -- start 2024-05-10T21:09:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.128+0000 7efe67600640 1 -- start start 2024-05-10T21:09:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 0x7efe6019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 0x7efe601a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efe6010f750 con 0x7efe60103c70 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe6010f5d0 con 0x7efe60104eb0 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe67600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe6010f8d0 con 0x7efe60109bb0 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46146/0 (socket says 172.21.15.50:46146) 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 0x7efe6019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.129+0000 7efe5fe00640 1 -- 172.21.15.50:0/305363105 learned_addr learned my addr 172.21.15.50:0/305363105 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe65600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 0x7efe601a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe5fe00640 1 -- 172.21.15.50:0/305363105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 msgr2=0x7efe601a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe5fe00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 0x7efe601a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe5fe00640 1 -- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 msgr2=0x7efe6019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe5fe00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 0x7efe6019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe5fe00640 1 -- 172.21.15.50:0/305363105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe601a4120 con 0x7efe60104eb0 2024-05-10T21:09:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.130+0000 7efe64c00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 0x7efe6019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe5fe00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e9f0 tx=0x7efe4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe5d600640 1 -- 172.21.15.50:0/305363105 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00cd80 con 0x7efe60104eb0 2024-05-10T21:09:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe601a4410 con 0x7efe60104eb0 2024-05-10T21:09:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe5d600640 1 -- 172.21.15.50:0/305363105 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4c0108e0 con 0x7efe60104eb0 2024-05-10T21:09:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe5d600640 1 -- 172.21.15.50:0/305363105 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c018980 con 0x7efe60104eb0 2024-05-10T21:09:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.131+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe601a4960 con 0x7efe60104eb0 2024-05-10T21:09:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.135+0000 7efe6e630740 1 Processor -- start 2024-05-10T21:09:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.135+0000 7efe6e630740 1 -- start start 2024-05-10T21:09:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113be92f0 0x561113fb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113bf14d0 0x561113fbb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561113f18550 con 0x561113be92f0 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561113ebdbc0 con 0x561113bf14d0 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe6e630740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561113f14510 con 0x561113c121b0 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113be92f0 0x561113fb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47362/0 (socket says 172.21.15.50:47362) 2024-05-10T21:09:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 -- 172.21.15.50:0/1062936547 learned_addr learned my addr 172.21.15.50:0/1062936547 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 -- 172.21.15.50:0/1062936547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113bf14d0 msgr2=0x561113fbb580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113bf14d0 0x561113fbb580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 -- 172.21.15.50:0/1062936547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113be92f0 msgr2=0x561113fb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113be92f0 0x561113fb4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.136+0000 7efe65600640 1 -- 172.21.15.50:0/1062936547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561113c2a210 con 0x561113c121b0 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.137+0000 7efe5d600640 1 -- 172.21.15.50:0/305363105 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efe4c010430 con 0x7efe60104eb0 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.137+0000 7efe65600640 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe580039f0 tx=0x7efe58003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae129661d46df5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.137+0000 7efe51c00640 1 -- 172.21.15.50:0/1062936547 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5801a470 con 0x561113c121b0 2024-05-10T21:09:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.137+0000 7efe51c00640 1 -- 172.21.15.50:0/1062936547 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe58015070 con 0x561113c121b0 2024-05-10T21:09:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.137+0000 7efe51c00640 1 -- 172.21.15.50:0/1062936547 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580107e0 con 0x561113c121b0 2024-05-10T21:09:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 -- 172.21.15.50:0/1062936547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 msgr2=0x561113fb4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe580039f0 tx=0x7efe58003eb0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 -- 172.21.15.50:0/1062936547 shutdown_connections 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113bf14d0 0x561113fbb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113be92f0 0x561113fb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 --2- 172.21.15.50:0/1062936547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113c121b0 0x561113fb4100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.138+0000 7efe6e630740 1 -- 172.21.15.50:0/1062936547 >> 172.21.15.50:0/1062936547 conn(0x561113c06300 msgr2=0x561113fafa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe6e630740 1 -- 172.21.15.50:0/1062936547 shutdown_connections 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe6e630740 1 -- 172.21.15.50:0/1062936547 wait complete. 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe5d600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 0x7efe480b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe5d600640 1 -- 172.21.15.50:0/305363105 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efe4c0e2560 con 0x7efe60104eb0 2024-05-10T21:09:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe64c00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 0x7efe480b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe6e630740 1 -- start start 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.139+0000 7efe6e630740 1 Processor -- start 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe64c00640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 0x7efe480b4c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efe54005ab0 tx=0x7efe54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fbe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561113fbbfd0 con 0x561113c121b0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561113ed1660 con 0x561113be92f0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561113fbc150 con 0x561113bf14d0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe6e630740 10 client.0 _ll_register_callbacks cb 0x561113e9b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe64c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe64c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46152/0 (socket says 172.21.15.50:46152) 2024-05-10T21:09:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe64c00640 1 -- 172.21.15.50:0/2771938543 learned_addr learned my addr 172.21.15.50:0/2771938543 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.140+0000 7efe65600640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe5fe00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fbe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 -- 172.21.15.50:0/2771938543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 msgr2=0x561113fbe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fbe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 -- 172.21.15.50:0/2771938543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 msgr2=0x561113fc50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 -- 172.21.15.50:0/2771938543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561113cf6620 con 0x561113be92f0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe65600640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe64c00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe60103540 tx=0x7efe54009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671759043f4e7925 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.141+0000 7efe46000640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54035040 con 0x561113be92f0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efe54003b80 con 0x561113be92f0 2024-05-10T21:09:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe58015070 con 0x561113be92f0 2024-05-10T21:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe54003d80 con 0x561113be92f0 2024-05-10T21:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe6e630740 10 client.25343 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561113fc8060 con 0x561113be92f0 2024-05-10T21:09:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe58015070 con 0x561113be92f0 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.142+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efe54044030 con 0x561113be92f0 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 10 client.25343 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 20 client.25343 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561113bd0d80 0x561113fcb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 4 client.25343 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561113f26160 con 0x561113bd0d80 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe5fe00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561113bd0d80 0x561113fcb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe46000640 10 client.25343 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.143+0000 7efe5fe00640 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561113bd0d80 0x561113fcb710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffe0aae5ee71678 server_cookie=7b1a212a74881c4d in_seq=0 out_seq=0 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.144+0000 7efe46000640 1 -- 172.21.15.50:0/2771938543 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561113f26160 con 0x561113bd0d80 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.144+0000 7efe46000640 10 client.25343 handle_command_reply: tid=0 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.144+0000 7efe6e630740 1 client.25343 shutdown 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 msgr2=0x561113fbdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe60103540 tx=0x7efe54009260 comp rx=0 tx=0).stop 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 shutdown_connections 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561113bd0d80 0x561113fcb710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fbe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 --2- 172.21.15.50:0/2771938543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdc20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe46000640 0 client.25343 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 >> 172.21.15.50:0/2771938543 conn(0x561113c06300 msgr2=0x561113faea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 shutdown_connections 2024-05-10T21:09:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 1 -- 172.21.15.50:0/2771938543 wait complete. 2024-05-10T21:09:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.145+0000 7efe6e630740 20 client.25343 trim_cache size 0 max 16384 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 Processor -- start 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 -- start start 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efe601afe50 con 0x561113c121b0 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe601ad720 con 0x561113be92f0 2024-05-10T21:09:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.162+0000 7efe6e630740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe601ab570 con 0x561113bf14d0 2024-05-10T21:09:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe65600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47384/0 (socket says 172.21.15.50:47384) 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe65600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59912/0 (socket says 172.21.15.50:59912) 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe5fe00640 1 -- 172.21.15.50:0/761742595 learned_addr learned my addr 172.21.15.50:0/761742595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 -- 172.21.15.50:0/761742595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 msgr2=0x561113fc5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 -- 172.21.15.50:0/761742595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 msgr2=0x561113fc5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe64c00640 1 -- 172.21.15.50:0/761742595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561113fab280 con 0x561113be92f0 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.163+0000 7efe5fe00640 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.164+0000 7efe64c00640 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdf00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe5403b590 tx=0x7efe5402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7318369228ee3a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.164+0000 7efe51200640 1 -- 172.21.15.50:0/761742595 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54035040 con 0x561113be92f0 2024-05-10T21:09:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.164+0000 7efe51200640 1 -- 172.21.15.50:0/761742595 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5404c070 con 0x561113be92f0 2024-05-10T21:09:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.164+0000 7efe51200640 1 -- 172.21.15.50:0/761742595 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540516b0 con 0x561113be92f0 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- 172.21.15.50:0/761742595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 msgr2=0x561113fbdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdf00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe5403b590 tx=0x7efe5402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- 172.21.15.50:0/761742595 shutdown_connections 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113bf14d0 0x561113fc5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113c121b0 0x561113fc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 --2- 172.21.15.50:0/761742595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113be92f0 0x561113fbdf00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- 172.21.15.50:0/761742595 >> 172.21.15.50:0/761742595 conn(0x561113c06300 msgr2=0x561113fbd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- 172.21.15.50:0/761742595 shutdown_connections 2024-05-10T21:09:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- 172.21.15.50:0/761742595 wait complete. 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.165+0000 7efe6e630740 1 -- start start 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 Processor -- start 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113c121b0 0x561113fc2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113ba61c0 0x561114031ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efe601abbe0 con 0x561113bf14d0 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe601ad050 con 0x561113c121b0 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe601ac2b0 con 0x561113ba61c0 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe6e630740 10 client.0 _ll_register_callbacks cb 0x7efe54032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe64c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59926/0 (socket says 172.21.15.50:59926) 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe64c00640 1 -- 172.21.15.50:0/3463885036 learned_addr learned my addr 172.21.15.50:0/3463885036 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.166+0000 7efe5fe00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113c121b0 0x561113fc2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe65600640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113ba61c0 0x561114031ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 -- 172.21.15.50:0/3463885036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113ba61c0 msgr2=0x561114031ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113ba61c0 0x561114031ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 -- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113c121b0 msgr2=0x561113fc2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113c121b0 0x561113fc2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 -- 172.21.15.50:0/3463885036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561113fb8ac0 con 0x561113bf14d0 2024-05-10T21:09:27.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:26 smithi050 ceph-mon[29546]: pgmap v477: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe64c00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efe54033000 tx=0x7efe5403ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8138ecb17bbe146 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.167+0000 7efe46000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54035040 con 0x561113bf14d0 2024-05-10T21:09:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efe54051680 con 0x561113bf14d0 2024-05-10T21:09:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5404c070 con 0x561113bf14d0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe540044d0 con 0x561113bf14d0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe6e630740 10 client.15234 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.168+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561114032ff0 con 0x561113bf14d0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5404c070 con 0x561113bf14d0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efe54044070 con 0x561113bf14d0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 10 client.15234 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 10 client.15234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 20 client.15234 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611140338c0 0x561114035ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 4 client.15234 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561113f1ffd0 con 0x5611140338c0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.169+0000 7efe5fe00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611140338c0 0x561114035ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.170+0000 7efe5fe00640 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611140338c0 0x561114035ce0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87023a652ab9e12 server_cookie=db9f9997a7be9fb5 in_seq=0 out_seq=0 2024-05-10T21:09:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.170+0000 7efe46000640 10 client.15234 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.174+0000 7efe46000640 1 -- 172.21.15.50:0/3463885036 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561113f1ffd0 con 0x5611140338c0 2024-05-10T21:09:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.174+0000 7efe46000640 10 client.15234 handle_command_reply: tid=0 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.174+0000 7efe6e630740 1 client.15234 shutdown 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 msgr2=0x561113fc2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efe54033000 tx=0x7efe5403ba30 comp rx=0 tx=0).stop 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 shutdown_connections 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611140338c0 0x561114035ce0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561113ba61c0 0x561114031ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561113c121b0 0x561113fc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe46000640 0 client.15234 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 --2- 172.21.15.50:0/3463885036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561113bf14d0 0x561113fc2570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 >> 172.21.15.50:0/3463885036 conn(0x561113c06300 msgr2=0x561113fb2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 shutdown_connections 2024-05-10T21:09:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 1 -- 172.21.15.50:0/3463885036 wait complete. 2024-05-10T21:09:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.175+0000 7efe6e630740 20 client.15234 trim_cache size 0 max 16384 2024-05-10T21:09:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:09:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:09:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:09:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 msgr2=0x7efe480b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 0x7efe480b4c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efe54005ab0 tx=0x7efe54000bc0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 msgr2=0x7efe6019cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e9f0 tx=0x7efe4c00eeb0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 shutdown_connections 2024-05-10T21:09:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efe480b26f0 0x7efe480b4c00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe60109bb0 0x7efe601a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe60104eb0 0x7efe6019cb20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 --2- 172.21.15.50:0/305363105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe60103c70 0x7efe6019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 >> 172.21.15.50:0/305363105 conn(0x7efe600fee60 msgr2=0x7efe600ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 shutdown_connections 2024-05-10T21:09:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:27.176+0000 7efe67600640 1 -- 172.21.15.50:0/305363105 wait complete. 2024-05-10T21:09:27.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:26 smithi092 ceph-mon[34415]: pgmap v477: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:26 smithi142 ceph-mon[34846]: pgmap v477: 97 pgs: 97 active+clean; 660 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T21:09:27.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:09:28.265 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:28 smithi050 ceph-mon[29546]: pgmap v478: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:29.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:28 smithi092 ceph-mon[34415]: pgmap v478: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:28 smithi142 ceph-mon[34846]: pgmap v478: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-10T21:09:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 -- 172.21.15.50:0/4072122354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 msgr2=0x7f74c810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 --2- 172.21.15.50:0/4072122354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c810ada0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009990 tx=0x7f74b4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 -- 172.21.15.50:0/4072122354 shutdown_connections 2024-05-10T21:09:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 --2- 172.21.15.50:0/4072122354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c810b2e0 0x7f74c8117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 --2- 172.21.15.50:0/4072122354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c8110970 0x7f74c8110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 --2- 172.21.15.50:0/4072122354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c810ada0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 -- 172.21.15.50:0/4072122354 >> 172.21.15.50:0/4072122354 conn(0x7f74c8077c80 msgr2=0x7f74c80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 -- 172.21.15.50:0/4072122354 shutdown_connections 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.619+0000 7f74cd400700 1 -- 172.21.15.50:0/4072122354 wait complete. 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 Processor -- start 2024-05-10T21:09:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 -- start start 2024-05-10T21:09:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c810b2e0 0x7f74c81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c8110970 0x7f74c81a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74c811aae0 con 0x7f74c810b2e0 2024-05-10T21:09:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74c811a960 con 0x7f74c810a990 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.621+0000 7f74cd400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74c811ac60 con 0x7f74c8110970 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46188/0 (socket says 172.21.15.50:46188) 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c6800700 1 -- 172.21.15.50:0/2195866238 learned_addr learned my addr 172.21.15.50:0/2195866238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c5e00700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c810b2e0 0x7f74c81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c7200700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c8110970 0x7f74c81a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.622+0000 7f74c6800700 1 -- 172.21.15.50:0/2195866238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c8110970 msgr2=0x7f74c81a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74c6800700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c8110970 0x7f74c81a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74c6800700 1 -- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c810b2e0 msgr2=0x7f74c81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74c6800700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c810b2e0 0x7f74c81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74c6800700 1 -- 172.21.15.50:0/2195866238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c81a6380 con 0x7f74c810a990 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74c6800700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74b800efc0 tx=0x7f74b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74bf400700 1 -- 172.21.15.50:0/2195866238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b800c9c0 con 0x7f74c810a990 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74bf400700 1 -- 172.21.15.50:0/2195866238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8019070 con 0x7f74c810a990 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74bf400700 1 -- 172.21.15.50:0/2195866238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8014480 con 0x7f74c810a990 2024-05-10T21:09:30.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.623+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c81a6610 con 0x7f74c810a990 2024-05-10T21:09:30.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.624+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c81a6b00 con 0x7f74c810a990 2024-05-10T21:09:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.628+0000 7f74bf400700 1 -- 172.21.15.50:0/2195866238 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f74b8012050 con 0x7f74c810a990 2024-05-10T21:09:30.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.629+0000 7f74dbfc4b80 1 Processor -- start 2024-05-10T21:09:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.629+0000 7f74dbfc4b80 1 -- start start 2024-05-10T21:09:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 0x55874f5724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55874f494a20 con 0x55874f578830 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55874f432760 con 0x55874f5729f0 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55874f4909e0 con 0x55874f3f4e40 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.630+0000 7f74bf400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 0x7f74ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 0x55874f5724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46200/0 (socket says 172.21.15.50:46200) 2024-05-10T21:09:30.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c5e00700 1 -- 172.21.15.50:0/975858966 learned_addr learned my addr 172.21.15.50:0/975858966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:30.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74bf400700 1 -- 172.21.15.50:0/2195866238 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74b80e2080 con 0x7f74c810a990 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c7200700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 0x7f74ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 -- 172.21.15.50:0/975858966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 msgr2=0x55874f5782f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5782f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 -- 172.21.15.50:0/975858966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 msgr2=0x55874f572e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c6800700 1 -- 172.21.15.50:0/975858966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55874f3e18d0 con 0x55874f578830 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.631+0000 7f74c5e00700 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.632+0000 7f74c6800700 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 0x55874f5724b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74b800cfd0 tx=0x7f74b809df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7d0b88050e46e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.632+0000 7f74c7200700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 0x7f74ac0b4e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74c00097a0 tx=0x7f74c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.632+0000 7f74b3e00700 1 -- 172.21.15.50:0/975858966 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a9180 con 0x55874f578830 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.632+0000 7f74b3e00700 1 -- 172.21.15.50:0/975858966 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80a9d70 con 0x55874f578830 2024-05-10T21:09:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.632+0000 7f74b3e00700 1 -- 172.21.15.50:0/975858966 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80b24d0 con 0x55874f578830 2024-05-10T21:09:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/975858966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 msgr2=0x55874f5724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 0x55874f5724b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74b800cfd0 tx=0x7f74b809df40 comp rx=0 tx=0).stop 2024-05-10T21:09:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/975858966 shutdown_connections 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f572e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/975858966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f578830 0x55874f5724b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/975858966 >> 172.21.15.50:0/975858966 conn(0x55874f439f70 msgr2=0x55874f56d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/975858966 shutdown_connections 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.633+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/975858966 wait complete. 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.634+0000 7f74dbfc4b80 1 -- start start 2024-05-10T21:09:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.634+0000 7f74dbfc4b80 1 Processor -- start 2024-05-10T21:09:30.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f57c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f57ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55874f57f2b0 con 0x55874f5729f0 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55874f3fab20 con 0x55874f578830 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55874f57f430 con 0x55874f3f4e40 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74dbfc4b80 10 client.0 _ll_register_callbacks cb 0x55874f43bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f57c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74c6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74c6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47438/0 (socket says 172.21.15.50:47438) 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74c6800700 1 -- 172.21.15.50:0/1954492156 learned_addr learned my addr 172.21.15.50:0/1954492156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.635+0000 7f74c7200700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f57ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 -- 172.21.15.50:0/1954492156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 msgr2=0x55874f57ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f57ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 -- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 msgr2=0x55874f57c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f57c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 -- 172.21.15.50:0/1954492156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874f45a420 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c7200700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f57ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74c6800700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74b80a9370 tx=0x7f74b8005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131d951429d6a90a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a9180 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74b80e40f0 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.636+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b80a9d70 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74b801a270 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 10 client.25354 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55874f57f2b0 con 0x55874f3f4e40 2024-05-10T21:09:30.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80a9d70 con 0x55874f3f4e40 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f74b80a98d0 con 0x55874f3f4e40 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 10 client.25354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 20 client.25354 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f58c0f0 0x55874f58e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 4 client.25354 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:30.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.637+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55874f53f1c0 con 0x55874f58c0f0 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.638+0000 7f74c5e00700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f58c0f0 0x55874f58e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.638+0000 7f74c5e00700 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f58c0f0 0x55874f58e510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd79111a38c3574c server_cookie=33747a9bfa69fd2c in_seq=0 out_seq=0 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.638+0000 7f74b0c00700 10 client.25354 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74b0c00700 1 -- 172.21.15.50:0/1954492156 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55874f53f1c0 con 0x55874f58c0f0 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74b0c00700 10 client.25354 handle_command_reply: tid=0 2024-05-10T21:09:30.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 client.25354 shutdown 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 msgr2=0x55874f57bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74b80a9370 tx=0x7f74b8005b70 comp rx=0 tx=0).stop 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 shutdown_connections 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f57ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f58c0f0 0x55874f58e510 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f57bff0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/1954492156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f57c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74b0c00700 0 client.25354 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.639+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 >> 172.21.15.50:0/1954492156 conn(0x55874f439f70 msgr2=0x55874f5714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.640+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 shutdown_connections 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.640+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/1954492156 wait complete. 2024-05-10T21:09:30.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.640+0000 7f74dbfc4b80 20 client.25354 trim_cache size 0 max 16384 2024-05-10T21:09:30.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 Processor -- start 2024-05-10T21:09:30.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 -- start start 2024-05-10T21:09:30.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f3f4e40 0x55874f579c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f578830 0x55874f57c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b7b20 con 0x55874f3f4e40 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b53b0 con 0x55874f5729f0 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b3240 con 0x55874f578830 2024-05-10T21:09:30.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.657+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c6800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f3f4e40 0x55874f579c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46212/0 (socket says 172.21.15.50:46212) 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 -- 172.21.15.50:0/642679714 learned_addr learned my addr 172.21.15.50:0/642679714 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c7200700 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f578830 0x55874f57c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 -- 172.21.15.50:0/642679714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f578830 msgr2=0x55874f57c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f578830 0x55874f57c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 -- 172.21.15.50:0/642679714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f3f4e40 msgr2=0x55874f579c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f3f4e40 0x55874f579c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 -- 172.21.15.50:0/642679714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55874f5890e0 con 0x55874f5729f0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.658+0000 7f74c5e00700 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009990 tx=0x7f74b4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6c039d87167a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.659+0000 7f74b3400700 1 -- 172.21.15.50:0/642679714 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b402e070 con 0x55874f5729f0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.659+0000 7f74b3400700 1 -- 172.21.15.50:0/642679714 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b400bbd0 con 0x55874f5729f0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.659+0000 7f74b3400700 1 -- 172.21.15.50:0/642679714 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4028170 con 0x55874f5729f0 2024-05-10T21:09:30.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.659+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/642679714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 msgr2=0x55874f5766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.659+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009990 tx=0x7f74b4005040 comp rx=0 tx=0).stop 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/642679714 shutdown_connections 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f578830 0x55874f57c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f3f4e40 0x55874f579c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/642679714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f5729f0 0x55874f5766e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/642679714 >> 172.21.15.50:0/642679714 conn(0x55874f439f70 msgr2=0x55874f577750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/642679714 shutdown_connections 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/642679714 wait complete. 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.660+0000 7f74dbfc4b80 1 -- start start 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 Processor -- start 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f593f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b3890 con 0x55874f5729f0 2024-05-10T21:09:30.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b4ce0 con 0x55874f578830 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74c81b3f40 con 0x55874f3f4e40 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.661+0000 7f74dbfc4b80 10 client.0 _ll_register_callbacks cb 0x55874f43bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59980/0 (socket says 172.21.15.50:59980) 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c5e00700 1 -- 172.21.15.50:0/66790981 learned_addr learned my addr 172.21.15.50:0/66790981 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c7200700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f593f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 -- 172.21.15.50:0/66790981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 msgr2=0x55874f593f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f593f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 -- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 msgr2=0x55874f5939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:30.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 -- 172.21.15.50:0/66790981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874f5f5770 con 0x55874f3f4e40 2024-05-10T21:09:30.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c5e00700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74c6800700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5934a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74b80a9600 tx=0x7f74b800cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618e793c1b3a6ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.662+0000 7f74b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a9180 con 0x55874f3f4e40 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b400bbd0 con 0x55874f3f4e40 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74b8005c00 con 0x55874f3f4e40 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74b80c8050 con 0x55874f3f4e40 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74dbfc4b80 10 client.25360 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55874f5f69d0 con 0x55874f3f4e40 2024-05-10T21:09:30.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.663+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b400bbd0 con 0x55874f3f4e40 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f74b80c0b70 con 0x55874f3f4e40 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 10 client.25360 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 10 client.25360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 20 client.25360 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f5f6e40 0x55874f5f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 4 client.25360 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55874f4a2400 con 0x55874f5f6e40 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.664+0000 7f74c5e00700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f5f6e40 0x55874f5f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74c5e00700 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f5f6e40 0x55874f5f9260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8cad8557759b814 server_cookie=cd399d074e059dab in_seq=0 out_seq=0 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74b0c00700 10 client.25360 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74b0c00700 1 -- 172.21.15.50:0/66790981 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55874f4a2400 con 0x55874f5f6e40 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74b0c00700 10 client.25360 handle_command_reply: tid=0 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 client.25360 shutdown 2024-05-10T21:09:30.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 msgr2=0x55874f5934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5934a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74b80a9600 tx=0x7f74b800cbd0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 shutdown_connections 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55874f578830 0x55874f593f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55874f5f6e40 0x55874f5f9260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55874f3f4e40 0x55874f5934a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 --2- 172.21.15.50:0/66790981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55874f5729f0 0x55874f5939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74b0c00700 0 client.25360 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.665+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 >> 172.21.15.50:0/66790981 conn(0x55874f439f70 msgr2=0x55874f5670a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 shutdown_connections 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74dbfc4b80 1 -- 172.21.15.50:0/66790981 wait complete. 2024-05-10T21:09:30.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74dbfc4b80 20 client.25360 trim_cache size 0 max 16384 2024-05-10T21:09:30.671 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:09:30.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 msgr2=0x7f74ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 0x7f74ac0b4e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74c00097a0 tx=0x7f74c0009260 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 msgr2=0x7f74c81a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.666+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74b800efc0 tx=0x7f74b800c3a0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 shutdown_connections 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74ac0b2900 0x7f74ac0b4e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c810a990 0x7f74c81a14b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c8110970 0x7f74c81a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 --2- 172.21.15.50:0/2195866238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74c810b2e0 0x7f74c81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 >> 172.21.15.50:0/2195866238 conn(0x7f74c8077c80 msgr2=0x7f74c807b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 shutdown_connections 2024-05-10T21:09:30.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:30.667+0000 7f74cd400700 1 -- 172.21.15.50:0/2195866238 wait complete. 2024-05-10T21:09:31.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:30 smithi092 ceph-mon[34415]: pgmap v479: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:09:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:30 smithi142 ceph-mon[34846]: pgmap v479: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:09:31.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:30 smithi050 ceph-mon[29546]: pgmap v479: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T21:09:31.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:09:32.062 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:33.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:32 smithi050 ceph-mon[29546]: pgmap v480: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:33.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:32 smithi092 ceph-mon[34415]: pgmap v480: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:33.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:32 smithi142 ceph-mon[34846]: pgmap v480: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 -- 172.21.15.50:0/3059217920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 msgr2=0x7f50a410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 --2- 172.21.15.50:0/3059217920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a410ada0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f50900098f0 tx=0x7f5090009c00 comp rx=0 tx=0).stop 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 -- 172.21.15.50:0/3059217920 shutdown_connections 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 --2- 172.21.15.50:0/3059217920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a410b2e0 0x7f50a4117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 --2- 172.21.15.50:0/3059217920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a4110970 0x7f50a4110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 --2- 172.21.15.50:0/3059217920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a410ada0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 -- 172.21.15.50:0/3059217920 >> 172.21.15.50:0/3059217920 conn(0x7f50a4077c80 msgr2=0x7f50a40780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 -- 172.21.15.50:0/3059217920 shutdown_connections 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.415+0000 7f50a9a00700 1 -- 172.21.15.50:0/3059217920 wait complete. 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.417+0000 7f50a9a00700 1 Processor -- start 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.417+0000 7f50a9a00700 1 -- start start 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a410b2e0 0x7f50a41a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a4110970 0x7f50a41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50a411aa90 con 0x7f50a4110970 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50a411a910 con 0x7f50a410a990 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a9a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50a411ac10 con 0x7f50a410b2e0 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:52402/0 (socket says 172.21.15.50:52402) 2024-05-10T21:09:34.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a2a00700 1 -- 172.21.15.50:0/84085834 learned_addr learned my addr 172.21.15.50:0/84085834 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.418+0000 7f50a2000700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a410b2e0 0x7f50a41a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a3400700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a4110970 0x7f50a41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 -- 172.21.15.50:0/84085834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a410b2e0 msgr2=0x7f50a41a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a410b2e0 0x7f50a41a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 -- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a4110970 msgr2=0x7f50a41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a4110970 0x7f50a41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 -- 172.21.15.50:0/84085834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a41a63d0 con 0x7f50a410a990 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a2a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f509b400700 1 -- 172.21.15.50:0/84085834 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400c9c0 con 0x7f50a410a990 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f509b400700 1 -- 172.21.15.50:0/84085834 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509400cb60 con 0x7f50a410a990 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a41a66c0 con 0x7f50a410a990 2024-05-10T21:09:34.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.419+0000 7f509b400700 1 -- 172.21.15.50:0/84085834 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094018680 con 0x7f50a410a990 2024-05-10T21:09:34.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.420+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41adfe0 con 0x7f50a410a990 2024-05-10T21:09:34.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.423+0000 7f509b400700 1 -- 172.21.15.50:0/84085834 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5094018a30 con 0x7f50a410a990 2024-05-10T21:09:34.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.424+0000 7f50b85a9b80 1 Processor -- start 2024-05-10T21:09:34.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 -- start start 2024-05-10T21:09:34.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f509b400700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 0x7f50880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196f2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c119613a20 con 0x55c119573e40 2024-05-10T21:09:34.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c1195b1760 con 0x55c1196f1c50 2024-05-10T21:09:34.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c11960f9e0 con 0x55c1196f7bf0 2024-05-10T21:09:34.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f509b400700 1 -- 172.21.15.50:0/84085834 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f50940e2570 con 0x7f50a410a990 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.425+0000 7f50a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56170/0 (socket says 172.21.15.50:56170) 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 -- 172.21.15.50:0/2691279866 learned_addr learned my addr 172.21.15.50:0/2691279866 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a2a00700 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a2000700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 0x7f50880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 -- 172.21.15.50:0/2691279866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 msgr2=0x55c1196f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 -- 172.21.15.50:0/2691279866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 msgr2=0x55c1196f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.426+0000 7f50a3400700 1 -- 172.21.15.50:0/2691279866 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1195608d0 con 0x55c119573e40 2024-05-10T21:09:34.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.427+0000 7f50a2000700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 0x7f50880b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5090009810 tx=0x7f5090019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.427+0000 7f50a3400700 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f509c007c50 tx=0x7f509c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d14230615eade0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.427+0000 7f508fe00700 1 -- 172.21.15.50:0/2691279866 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c020070 con 0x55c119573e40 2024-05-10T21:09:34.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.427+0000 7f508fe00700 1 -- 172.21.15.50:0/2691279866 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c017cb0 con 0x55c119573e40 2024-05-10T21:09:34.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.427+0000 7f508fe00700 1 -- 172.21.15.50:0/2691279866 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c01bba0 con 0x55c119573e40 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 -- 172.21.15.50:0/2691279866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 msgr2=0x55c1196f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f509c007c50 tx=0x7f509c009920 comp rx=0 tx=0).stop 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 -- 172.21.15.50:0/2691279866 shutdown_connections 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196f76b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/2691279866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 -- 172.21.15.50:0/2691279866 >> 172.21.15.50:0/2691279866 conn(0x55c1195b2a80 msgr2=0x55c1196ec790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.428+0000 7f50b85a9b80 1 -- 172.21.15.50:0/2691279866 shutdown_connections 2024-05-10T21:09:34.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.429+0000 7f50b85a9b80 1 -- 172.21.15.50:0/2691279866 wait complete. 2024-05-10T21:09:34.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.429+0000 7f50b85a9b80 1 -- start start 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.429+0000 7f50b85a9b80 1 Processor -- start 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196fb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c1196fe660 con 0x55c119573e40 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c119579b20 con 0x55c1196f1c50 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c1196fe7e0 con 0x55c1196f7bf0 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50b85a9b80 10 client.0 _ll_register_callbacks cb 0x55c1195a7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50a3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196fb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56176/0 (socket says 172.21.15.50:56176) 2024-05-10T21:09:34.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.430+0000 7f50a2a00700 1 -- 172.21.15.50:0/3398268983 learned_addr learned my addr 172.21.15.50:0/3398268983 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:34.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a2000700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 -- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 msgr2=0x55c1196fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 -- 172.21.15.50:0/3398268983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 msgr2=0x55c1196fb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196fb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 -- 172.21.15.50:0/3398268983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1195d9420 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f50a3400700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196fbe50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509c01e040 tx=0x7f509c0093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e3eda8a484286 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.431+0000 7f508cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c020040 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f509c003c00 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f509c017cb0 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f509c009e60 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f50b85a9b80 10 client.25372 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c119709280 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.432+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c017cb0 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f509c012030 con 0x55c1196f7bf0 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 10 client.25372 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 10 client.25372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:34.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 20 client.25372 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:34.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c11970b7b0 0x55c11970dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 4 client.25372 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:34.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1196be070 con 0x55c11970b7b0 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.433+0000 7f50a2a00700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c11970b7b0 0x55c11970dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.434+0000 7f50a2a00700 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c11970b7b0 0x55c11970dbd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be0b32f0aa74405 server_cookie=3aac8c0ac18adbaa in_seq=0 out_seq=0 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.434+0000 7f508cc00700 10 client.25372 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f508cc00700 1 -- 172.21.15.50:0/3398268983 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c1196be070 con 0x55c11970b7b0 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f508cc00700 10 client.25372 handle_command_reply: tid=0 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 client.25372 shutdown 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 msgr2=0x55c1196fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196fbe50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509c01e040 tx=0x7f509c0093f0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 shutdown_connections 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c11970b7b0 0x55c11970dbd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f7bf0 0x55c1196fbe50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c119573e40 0x55c1196fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f508cc00700 0 client.25372 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/3398268983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c1196fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 >> 172.21.15.50:0/3398268983 conn(0x55c1195b2a80 msgr2=0x55c1196f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 shutdown_connections 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.435+0000 7f50b85a9b80 1 -- 172.21.15.50:0/3398268983 wait complete. 2024-05-10T21:09:34.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.436+0000 7f50b85a9b80 20 client.25372 trim_cache size 0 max 16384 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.452+0000 7f50b85a9b80 1 Processor -- start 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.452+0000 7f50b85a9b80 1 -- start start 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 0x55c1196f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c1196fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b7a50 con 0x55c1196f7bf0 2024-05-10T21:09:34.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b52e0 con 0x55c119573e40 2024-05-10T21:09:34.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b31b0 con 0x55c1196f1c50 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c1196fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:52414/0 (socket says 172.21.15.50:52414) 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c1196fa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56182/0 (socket says 172.21.15.50:56182) 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a2a00700 1 -- 172.21.15.50:0/1059663678 learned_addr learned my addr 172.21.15.50:0/1059663678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.453+0000 7f50a2000700 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 0x55c1196f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 -- 172.21.15.50:0/1059663678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 msgr2=0x55c1196f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 0x55c1196f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 -- 172.21.15.50:0/1059663678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 msgr2=0x55c1196fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c1196fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 -- 172.21.15.50:0/1059663678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1196e8d90 con 0x55c119573e40 2024-05-10T21:09:34.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2000700 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 0x55c1196f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:34.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50940a9af0 tx=0x7f5094010b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b75809a50c8030b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f508f400700 1 -- 172.21.15.50:0/1059663678 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940a9b60 con 0x55c119573e40 2024-05-10T21:09:34.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f508f400700 1 -- 172.21.15.50:0/1059663678 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50940adf10 con 0x55c119573e40 2024-05-10T21:09:34.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.454+0000 7f508f400700 1 -- 172.21.15.50:0/1059663678 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940e20b0 con 0x55c119573e40 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1059663678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 msgr2=0x55c1196f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50940a9af0 tx=0x7f5094010b60 comp rx=0 tx=0).stop 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1059663678 shutdown_connections 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c119573e40 0x55c1196f1710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c1196fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1059663678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1196f1c50 0x55c1196f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1059663678 >> 172.21.15.50:0/1059663678 conn(0x55c1195b2a80 msgr2=0x55c1196f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1059663678 shutdown_connections 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.455+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1059663678 wait complete. 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.456+0000 7f50b85a9b80 1 -- start start 2024-05-10T21:09:34.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.456+0000 7f50b85a9b80 1 Processor -- start 2024-05-10T21:09:34.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c11970e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c119773cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b3800 con 0x55c1196f7bf0 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b4c10 con 0x55c1196f1c50 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50a41b3e70 con 0x55c119573e40 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50b85a9b80 10 client.0 _ll_register_callbacks cb 0x55c1195a7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c119773cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50a2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58702/0 (socket says 172.21.15.50:58702) 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50a2a00700 1 -- 172.21.15.50:0/1114916178 learned_addr learned my addr 172.21.15.50:0/1114916178 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.457+0000 7f50a2000700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c11970e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 -- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 msgr2=0x55c11970e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c11970e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 -- 172.21.15.50:0/1114916178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 msgr2=0x55c119773cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c119773cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 -- 172.21.15.50:0/1114916178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c119774260 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a3400700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c119773cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50a2a00700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f50940b2ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53937e0aa269b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f508cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940b2730 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f50940c41c0 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.458+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50940adf10 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50940e20b0 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 10 client.25378 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1197754c0 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50940adf10 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f50940a8830 con 0x55c119573e40 2024-05-10T21:09:34.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:34.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:34.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 20 client.25378 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:34.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c119775a90 0x55c119777eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:34.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 4 client.25378 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:09:34.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.459+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c119621400 con 0x55c119775a90 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.460+0000 7f50a2000700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c119775a90 0x55c119777eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.460+0000 7f50a2000700 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c119775a90 0x55c119777eb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2f3414194da311 server_cookie=f34461194f77e98a in_seq=0 out_seq=0 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.460+0000 7f508cc00700 10 client.25378 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f508cc00700 1 -- 172.21.15.50:0/1114916178 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c119621400 con 0x55c119775a90 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f508cc00700 10 client.25378 handle_command_reply: tid=0 2024-05-10T21:09:34.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 client.25378 shutdown 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 msgr2=0x55c11970df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f50940b2ab0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 shutdown_connections 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c119775a90 0x55c119777eb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c1196f7bf0 0x55c119773cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c119573e40 0x55c11970df90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 --2- 172.21.15.50:0/1114916178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c1196f1c50 0x55c11970e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f508cc00700 0 client.25378 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 >> 172.21.15.50:0/1114916178 conn(0x55c1195b2a80 msgr2=0x55c1196ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 shutdown_connections 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 1 -- 172.21.15.50:0/1114916178 wait complete. 2024-05-10T21:09:34.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.461+0000 7f50b85a9b80 20 client.25378 trim_cache size 0 max 16384 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 msgr2=0x7f50880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 0x7f50880b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5090009810 tx=0x7f5090019040 comp rx=0 tx=0).stop 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 msgr2=0x7f50a41a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400ed00 comp rx=0 tx=0).stop 2024-05-10T21:09:34.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 shutdown_connections 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50880b2900 0x7f50880b4e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50a410a990 0x7f50a41a1500 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50a4110970 0x7f50a41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 --2- 172.21.15.50:0/84085834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a410b2e0 0x7f50a41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 >> 172.21.15.50:0/84085834 conn(0x7f50a4077c80 msgr2=0x7f50a407b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 shutdown_connections 2024-05-10T21:09:34.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:34.462+0000 7f50a9a00700 1 -- 172.21.15.50:0/84085834 wait complete. 2024-05-10T21:09:35.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:34 smithi050 ceph-mon[29546]: pgmap v481: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:34 smithi092 ceph-mon[34415]: pgmap v481: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:35.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:34 smithi142 ceph-mon[34846]: pgmap v481: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:36.366 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:09:36.367 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:09:36.849 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:37.201 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:36 smithi050 ceph-mon[29546]: pgmap v482: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:37.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:36 smithi092 ceph-mon[34415]: pgmap v482: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:36 smithi142 ceph-mon[34846]: pgmap v482: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:09:39.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 -- 172.21.15.50:0/2128855578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 msgr2=0x7f45bc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 --2- 172.21.15.50:0/2128855578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc109330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009920 tx=0x7f45a8009c30 comp rx=0 tx=0).stop 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 -- 172.21.15.50:0/2128855578 shutdown_connections 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 --2- 172.21.15.50:0/2128855578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc109870 0x7f45bc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 --2- 172.21.15.50:0/2128855578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc1045f0 0x7f45bc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 --2- 172.21.15.50:0/2128855578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc109330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 -- 172.21.15.50:0/2128855578 >> 172.21.15.50:0/2128855578 conn(0x7f45bc100040 msgr2=0x7f45bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.094+0000 7f45c4400700 1 -- 172.21.15.50:0/2128855578 shutdown_connections 2024-05-10T21:09:39.095 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:38 smithi050 ceph-mon[29546]: pgmap v483: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:39.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.095+0000 7f45c4400700 1 -- 172.21.15.50:0/2128855578 wait complete. 2024-05-10T21:09:39.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.096+0000 7f45c4400700 1 Processor -- start 2024-05-10T21:09:39.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.096+0000 7f45c4400700 1 -- start start 2024-05-10T21:09:39.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 0x7f45bc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 0x7f45bc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1144d0 con 0x7f45bc109870 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc114350 con 0x7f45bc1045f0 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c4400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc114650 con 0x7f45bc108f20 2024-05-10T21:09:39.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 0x7f45bc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 0x7f45bc075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:52432/0 (socket says 172.21.15.50:52432) 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.097+0000 7f45c1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1a00700 1 -- 172.21.15.50:0/3408709082 learned_addr learned my addr 172.21.15.50:0/3408709082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c2400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 0x7f45bc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 -- 172.21.15.50:0/3408709082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 msgr2=0x7f45bc075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 0x7f45bc075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 -- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 msgr2=0x7f45bc076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 0x7f45bc076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 -- 172.21.15.50:0/3408709082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1a9380 con 0x7f45bc108f20 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c2400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 0x7f45bc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.098+0000 7f45c1000700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc076000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45a8000c00 tx=0x7f45a800f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.099+0000 7f45ba000700 1 -- 172.21.15.50:0/3408709082 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8027070 con 0x7f45bc108f20 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.099+0000 7f45ba000700 1 -- 172.21.15.50:0/3408709082 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a8005570 con 0x7f45bc108f20 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.099+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1a9610 con 0x7f45bc108f20 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.099+0000 7f45ba000700 1 -- 172.21.15.50:0/3408709082 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004930 con 0x7f45bc108f20 2024-05-10T21:09:39.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.099+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1a9a60 con 0x7f45bc108f20 2024-05-10T21:09:39.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.102+0000 7f45ba000700 1 -- 172.21.15.50:0/3408709082 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f45a802a070 con 0x7f45bc108f20 2024-05-10T21:09:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 Processor -- start 2024-05-10T21:09:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 -- start start 2024-05-10T21:09:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934add370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934ad7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45ba000700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 0x7f45a40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934ad7a70 0x559934ad7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5599349f9aa0 con 0x559934959ec0 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5599349977e0 con 0x559934ad7a70 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45d3041b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5599349f5a60 con 0x559934add8b0 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.104+0000 7f45c2400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934add370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45ba000700 1 -- 172.21.15.50:0/3408709082 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f45a80e2b60 con 0x7f45bc108f20 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c2400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934add370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56222/0 (socket says 172.21.15.50:56222) 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c2400700 1 -- 172.21.15.50:0/2475884033 learned_addr learned my addr 172.21.15.50:0/2475884033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 0x7f45a40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934ad7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c2400700 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934ad7a70 0x559934ad7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 -- 172.21.15.50:0/2475884033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934ad7a70 msgr2=0x559934ad7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934ad7a70 0x559934ad7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 -- 172.21.15.50:0/2475884033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 msgr2=0x559934add370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934add370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.105+0000 7f45c1000700 1 -- 172.21.15.50:0/2475884033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559934946950 con 0x559934add8b0 2024-05-10T21:09:39.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.106+0000 7f45c1000700 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934ad7530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45a80a9ee0 tx=0x7f45a80a61d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfae52a8bcb383b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.106+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 0x7f45a40b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005950 tx=0x7f45b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.106+0000 7f45aea00700 1 -- 172.21.15.50:0/2475884033 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80a9cf0 con 0x559934add8b0 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.106+0000 7f45aea00700 1 -- 172.21.15.50:0/2475884033 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a80aa8e0 con 0x559934add8b0 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.106+0000 7f45aea00700 1 -- 172.21.15.50:0/2475884033 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80b2fb0 con 0x559934add8b0 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 -- 172.21.15.50:0/2475884033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 msgr2=0x559934ad7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934ad7530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45a80a9ee0 tx=0x7f45a80a61d0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 -- 172.21.15.50:0/2475884033 shutdown_connections 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934ad7a70 0x559934ad7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934add370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2475884033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934ad7530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 -- 172.21.15.50:0/2475884033 >> 172.21.15.50:0/2475884033 conn(0x55993499eff0 msgr2=0x559934ad2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 -- 172.21.15.50:0/2475884033 shutdown_connections 2024-05-10T21:09:39.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.107+0000 7f45d3041b80 1 -- 172.21.15.50:0/2475884033 wait complete. 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.108+0000 7f45d3041b80 1 -- start start 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.108+0000 7f45d3041b80 1 Processor -- start 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934ae1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 0x559934ae1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934ae1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559934ae4150 con 0x559934959ec0 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55993495fba0 con 0x559934add8b0 2024-05-10T21:09:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559934ae42d0 con 0x559934ad7a70 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45d3041b80 10 client.0 _ll_register_callbacks cb 0x5599349a0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934ae1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934ae1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56228/0 (socket says 172.21.15.50:56228) 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45c1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 0x559934ae1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45c1a00700 1 -- 172.21.15.50:0/2615004989 learned_addr learned my addr 172.21.15.50:0/2615004989 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.109+0000 7f45c2400700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934ae1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 -- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 msgr2=0x559934ae1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934ae1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 -- 172.21.15.50:0/2615004989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 msgr2=0x559934ae1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934ae1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 -- 172.21.15.50:0/2615004989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599349bf4a0 con 0x559934ad7a70 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c2400700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934ae1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45a3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45c1000700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 0x559934ae1540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45a80aaa90 tx=0x7f45a80057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2297c121ab009e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80a9cf0 con 0x559934ad7a70 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a80aa8e0 con 0x559934ad7a70 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.110+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f45a80ae120 con 0x559934ad7a70 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45a80b2fb0 con 0x559934ad7a70 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 10 client.25390 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:39.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559934ae4150 con 0x559934ad7a70 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a80aa8e0 con 0x559934ad7a70 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f45a80b2b20 con 0x559934ad7a70 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 20 client.25390 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934af0e40 0x559934af3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 4 client.25390 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.111+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559934aa4240 con 0x559934af0e40 2024-05-10T21:09:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.112+0000 7f45c1a00700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934af0e40 0x559934af3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.112+0000 7f45c1a00700 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934af0e40 0x559934af3260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5543c91c592845 server_cookie=2b308461cfc4d848 in_seq=0 out_seq=0 2024-05-10T21:09:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.112+0000 7f45a3400700 10 client.25390 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45a3400700 1 -- 172.21.15.50:0/2615004989 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559934aa4240 con 0x559934af0e40 2024-05-10T21:09:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45a3400700 10 client.25390 handle_command_reply: tid=0 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 client.25390 shutdown 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 msgr2=0x559934ae1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 0x559934ae1540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45a80aaa90 tx=0x7f45a80057e0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 shutdown_connections 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934af0e40 0x559934af3260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934ae1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934959ec0 0x559934ae1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45a3400700 0 client.25390 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.113+0000 7f45d3041b80 1 --2- 172.21.15.50:0/2615004989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934ad7a70 0x559934ae1540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.114+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 >> 172.21.15.50:0/2615004989 conn(0x55993499eff0 msgr2=0x559934ad6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.114+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 shutdown_connections 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.114+0000 7f45d3041b80 1 -- 172.21.15.50:0/2615004989 wait complete. 2024-05-10T21:09:39.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.114+0000 7f45d3041b80 20 client.25390 trim_cache size 0 max 16384 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.136+0000 7f45d3041b80 1 Processor -- start 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.136+0000 7f45d3041b80 1 -- start start 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934adcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934aed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b34b0 con 0x559934ad7a70 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b0d40 con 0x559934959ec0 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45d3041b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1aebb0 con 0x559934add8b0 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45c1000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934adcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.137+0000 7f45c1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:52454/0 (socket says 172.21.15.50:52454) 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 -- 172.21.15.50:0/451547629 learned_addr learned my addr 172.21.15.50:0/451547629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:39.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c2400700 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934aed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 -- 172.21.15.50:0/451547629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 msgr2=0x559934aed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934aed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 -- 172.21.15.50:0/451547629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 msgr2=0x559934adcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934adcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 -- 172.21.15.50:0/451547629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559934accea0 con 0x559934959ec0 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.138+0000 7f45c1a00700 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x559934af3800 tx=0x7f45b0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a85372f15e6ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.139+0000 7f45ae000700 1 -- 172.21.15.50:0/451547629 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b002d070 con 0x559934959ec0 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.139+0000 7f45ae000700 1 -- 172.21.15.50:0/451547629 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b0003bb0 con 0x559934959ec0 2024-05-10T21:09:39.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.139+0000 7f45ae000700 1 -- 172.21.15.50:0/451547629 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0027ad0 con 0x559934959ec0 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 -- 172.21.15.50:0/451547629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 msgr2=0x559934ad7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x559934af3800 tx=0x7f45b0014000 comp rx=0 tx=0).stop 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 -- 172.21.15.50:0/451547629 shutdown_connections 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934959ec0 0x559934ad7530 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934add8b0 0x559934aed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 --2- 172.21.15.50:0/451547629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934adcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 -- 172.21.15.50:0/451547629 >> 172.21.15.50:0/451547629 conn(0x55993499eff0 msgr2=0x559934acc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 -- 172.21.15.50:0/451547629 shutdown_connections 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.140+0000 7f45d3041b80 1 -- 172.21.15.50:0/451547629 wait complete. 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.141+0000 7f45d3041b80 1 -- start start 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.141+0000 7f45d3041b80 1 Processor -- start 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 0x559934ae66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934b5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1af200 con 0x559934ad7a70 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b0670 con 0x559934add8b0 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1af8d0 con 0x559934959ec0 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45d3041b80 10 client.0 _ll_register_callbacks cb 0x5599349a0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56244/0 (socket says 172.21.15.50:56244) 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1000700 1 -- 172.21.15.50:0/3071185864 learned_addr learned my addr 172.21.15.50:0/3071185864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 0x559934ae66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c2400700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934b5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 -- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 msgr2=0x559934b5b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934b5b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 -- 172.21.15.50:0/3071185864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 msgr2=0x559934ae6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1a00700 1 -- 172.21.15.50:0/3071185864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559934b5b960 con 0x559934959ec0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.142+0000 7f45c1000700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45c1a00700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 0x559934ae66e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45b002b040 tx=0x7f45b0003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30c6a0e1bde96e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45a3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b002d070 con 0x559934959ec0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f45b00273e0 con 0x559934959ec0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b0003bb0 con 0x559934959ec0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45b0022040 con 0x559934959ec0 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45d3041b80 10 client.25396 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:39.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.143+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559934b5cad0 con 0x559934959ec0 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b0003bb0 con 0x559934959ec0 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f45b0027ce0 con 0x559934959ec0 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 10 client.25396 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 20 client.25396 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934b5d0c0 0x559934b5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:39.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 4 client.25396 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.144+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559934a07480 con 0x559934b5d0c0 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.145+0000 7f45c1000700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934b5d0c0 0x559934b5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.145+0000 7f45c1000700 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934b5d0c0 0x559934b5f4e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d309a932f273d44 server_cookie=b522b0521c43b4b1 in_seq=0 out_seq=0 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.145+0000 7f45a3400700 10 client.25396 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45a3400700 1 -- 172.21.15.50:0/3071185864 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559934a07480 con 0x559934b5d0c0 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45a3400700 10 client.25396 handle_command_reply: tid=0 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 client.25396 shutdown 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 msgr2=0x559934ae66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 0x559934ae66e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45b002b040 tx=0x7f45b0003f60 comp rx=0 tx=0).stop 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 shutdown_connections 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559934b5d0c0 0x559934b5f4e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559934add8b0 0x559934b5b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559934959ec0 0x559934ae66e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 --2- 172.21.15.50:0/3071185864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559934ad7a70 0x559934ae6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 >> 172.21.15.50:0/3071185864 conn(0x55993499eff0 msgr2=0x559934acc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45a3400700 0 client.25396 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 shutdown_connections 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 1 -- 172.21.15.50:0/3071185864 wait complete. 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.147+0000 7f45d3041b80 20 client.25396 trim_cache size 0 max 16384 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "125fcf26-87e2-491d-b227-fbe731b7734a", 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:09:39.149 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:39.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.148+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 msgr2=0x7f45a40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.148+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 0x7f45a40b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005950 tx=0x7f45b0009310 comp rx=0 tx=0).stop 2024-05-10T21:09:39.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.148+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 msgr2=0x7f45bc076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.148+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc076000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45a8000c00 tx=0x7f45a800f950 comp rx=0 tx=0).stop 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 shutdown_connections 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f45a40b2900 0x7f45a40b4e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc1045f0 0x7f45bc075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45bc109870 0x7f45bc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 --2- 172.21.15.50:0/3408709082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45bc108f20 0x7f45bc076000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 >> 172.21.15.50:0/3408709082 conn(0x7f45bc100040 msgr2=0x7f45bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:39.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 shutdown_connections 2024-05-10T21:09:39.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:39.149+0000 7f45c4400700 1 -- 172.21.15.50:0/3408709082 wait complete. 2024-05-10T21:09:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:38 smithi092 ceph-mon[34415]: pgmap v483: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:39.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:38 smithi142 ceph-mon[34846]: pgmap v483: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:39.551 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-10T21:09:39.551 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.35 2024-05-10T21:09:39.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:09:40.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:09:40.148 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:40.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:39 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:09:40.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:39 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:39 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:09:40.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:39 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:39 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:09:40.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:39 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:09:40.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:39 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:39 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:09:40.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:39 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:39 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:09:40.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:39 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:09:40.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:39 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:39 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:09:40.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:39 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:09:40.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:39 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:09:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 -- 172.21.15.50:0/2310799549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec109a00 msgr2=0x7f43ec10be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 --2- 172.21.15.50:0/2310799549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec109a00 0x7f43ec10be20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009ca0 tx=0x7f43e402a8c0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 -- 172.21.15.50:0/2310799549 shutdown_connections 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 --2- 172.21.15.50:0/2310799549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec109a00 0x7f43ec10be20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 --2- 172.21.15.50:0/2310799549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec1070a0 0x7f43ec1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 --2- 172.21.15.50:0/2310799549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec104740 0x7f43ec106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 -- 172.21.15.50:0/2310799549 >> 172.21.15.50:0/2310799549 conn(0x7f43ec0fdd40 msgr2=0x7f43ec1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 -- 172.21.15.50:0/2310799549 shutdown_connections 2024-05-10T21:09:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.398+0000 7f43f1600640 1 -- 172.21.15.50:0/2310799549 wait complete. 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.399+0000 7f43f1600640 1 Processor -- start 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.399+0000 7f43f1600640 1 -- start start 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 0x7f43ec19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec109a00 0x7f43ec1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10f7b0 con 0x7f43ec1070a0 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10f630 con 0x7f43ec109a00 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43f1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec10f930 con 0x7f43ec104740 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56260/0 (socket says 172.21.15.50:56260) 2024-05-10T21:09:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43e3e00640 1 -- 172.21.15.50:0/465253501 learned_addr learned my addr 172.21.15.50:0/465253501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43eaa00640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 0x7f43ec19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.400+0000 7f43eb400640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec109a00 0x7f43ec1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 -- 172.21.15.50:0/465253501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 msgr2=0x7f43ec19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 0x7f43ec19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 -- 172.21.15.50:0/465253501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec109a00 msgr2=0x7f43ec1a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec109a00 0x7f43ec1a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 -- 172.21.15.50:0/465253501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec1a4090 con 0x7f43ec1070a0 2024-05-10T21:09:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43eaa00640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 0x7f43ec19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3e00640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e9f0 tx=0x7f43d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3400640 1 -- 172.21.15.50:0/465253501 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d400cd80 con 0x7f43ec1070a0 2024-05-10T21:09:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec1a4380 con 0x7f43ec1070a0 2024-05-10T21:09:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.401+0000 7f43e3400640 1 -- 172.21.15.50:0/465253501 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d40108e0 con 0x7f43ec1070a0 2024-05-10T21:09:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.402+0000 7f43e3400640 1 -- 172.21.15.50:0/465253501 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4018980 con 0x7f43ec1070a0 2024-05-10T21:09:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.402+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1a4920 con 0x7f43ec1070a0 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.405+0000 7f43f862d740 1 Processor -- start 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.405+0000 7f43f862d740 1 -- start start 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe37100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 0x55e1cfe3e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfd9b550 con 0x55e1cfa744d0 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfd40bc0 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43f862d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfd97510 con 0x55e1cfa951b0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43e3400640 1 -- 172.21.15.50:0/465253501 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f43d4010430 con 0x7f43ec1070a0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:52482/0 (socket says 172.21.15.50:52482) 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eaa00640 1 -- 172.21.15.50:0/2405859895 learned_addr learned my addr 172.21.15.50:0/2405859895 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eb400640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe37100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43e3e00640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 0x55e1cfe3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eaa00640 1 -- 172.21.15.50:0/2405859895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 msgr2=0x55e1cfe37100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.406+0000 7f43eaa00640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe37100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43eaa00640 1 -- 172.21.15.50:0/2405859895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 msgr2=0x55e1cfe3e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43eaa00640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 0x55e1cfe3e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43eaa00640 1 -- 172.21.15.50:0/2405859895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cfaad210 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43e3e00640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 0x55e1cfe3e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:09:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43eaa00640 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0114f0 tx=0x7f43dc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cec11e569665ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43db400640 1 -- 172.21.15.50:0/2405859895 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc01e070 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43db400640 1 -- 172.21.15.50:0/2405859895 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc0035f0 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.407+0000 7f43db400640 1 -- 172.21.15.50:0/2405859895 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00f820 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43e3400640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 0x7f43d00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43e3400640 1 -- 172.21.15.50:0/465253501 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f43d40e2540 con 0x7f43ec1070a0 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43eb400640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 0x7f43d00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 -- 172.21.15.50:0/2405859895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 msgr2=0x55e1cfe37680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0114f0 tx=0x7f43dc0119b0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 -- 172.21.15.50:0/2405859895 shutdown_connections 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa744d0 0x55e1cfe3e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe37680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 --2- 172.21.15.50:0/2405859895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe37100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 -- 172.21.15.50:0/2405859895 >> 172.21.15.50:0/2405859895 conn(0x55e1cfa89300 msgr2=0x55e1cfe32a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.408+0000 7f43f862d740 1 -- 172.21.15.50:0/2405859895 shutdown_connections 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.409+0000 7f43f862d740 1 -- 172.21.15.50:0/2405859895 wait complete. 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.409+0000 7f43eb400640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 0x7f43d00b4cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005e30 tx=0x7f43e4031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.409+0000 7f43f862d740 1 -- start start 2024-05-10T21:09:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.409+0000 7f43f862d740 1 Processor -- start 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe40ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfe3efd0 con 0x55e1cfa951b0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfd54660 con 0x55e1cfa744d0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e1cfe3f150 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43f862d740 10 client.0 _ll_register_callbacks cb 0x55e1cfd1e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56288/0 (socket says 172.21.15.50:56288) 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.410+0000 7f43eb400640 1 -- 172.21.15.50:0/1302176459 learned_addr learned my addr 172.21.15.50:0/1302176459 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43e3e00640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe40ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 -- 172.21.15.50:0/1302176459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 msgr2=0x55e1cfe40ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe40ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 -- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 msgr2=0x55e1cfe410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 -- 172.21.15.50:0/1302176459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cfb79620 con 0x55e1cfa951b0 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43e3e00640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43eb400640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43e4035000 tx=0x7f43e402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6856f19d26c6e1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.411+0000 7f43cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4004250 con 0x55e1cfa951b0 2024-05-10T21:09:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc0035f0 con 0x55e1cfa951b0 2024-05-10T21:09:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f43e40043f0 con 0x55e1cfa951b0 2024-05-10T21:09:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43e40045f0 con 0x55e1cfa951b0 2024-05-10T21:09:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43f862d740 10 client.15297 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1cfe4aee0 con 0x55e1cfa951b0 2024-05-10T21:09:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc0035f0 con 0x55e1cfa951b0 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.412+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43e4051070 con 0x55e1cfa951b0 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 10 client.15297 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 20 client.15297 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa53d80 0x55e1cfe4e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 4 client.15297 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1cfda9160 con 0x55e1cfa53d80 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.413+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa53d80 0x55e1cfe4e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.414+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa53d80 0x55e1cfe4e700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a66fc4ed4e3c5c2 server_cookie=2f393157eac8ff2d in_seq=0 out_seq=0 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.414+0000 7f43cfe00640 10 client.15297 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.414+0000 7f43cfe00640 1 -- 172.21.15.50:0/1302176459 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e1cfda9160 con 0x55e1cfa53d80 2024-05-10T21:09:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.414+0000 7f43cfe00640 10 client.15297 handle_command_reply: tid=0 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.414+0000 7f43f862d740 1 client.15297 shutdown 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 msgr2=0x55e1cfe48040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43e4035000 tx=0x7f43e402a8c0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 shutdown_connections 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa53d80 0x55e1cfe4e700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe48040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 --2- 172.21.15.50:0/1302176459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe40ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43cfe00640 0 client.15297 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 >> 172.21.15.50:0/1302176459 conn(0x55e1cfa89300 msgr2=0x55e1cfe359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 shutdown_connections 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 1 -- 172.21.15.50:0/1302176459 wait complete. 2024-05-10T21:09:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.415+0000 7f43f862d740 20 client.15297 trim_cache size 0 max 16384 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.432+0000 7f43f862d740 1 Processor -- start 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.432+0000 7f43f862d740 1 -- start start 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe32990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1afd80 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ad650 con 0x55e1cfa744d0 2024-05-10T21:09:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43f862d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ab4a0 con 0x55e1cfa951b0 2024-05-10T21:09:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43eb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56294/0 (socket says 172.21.15.50:56294) 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43eaa00640 1 -- 172.21.15.50:0/1268446669 learned_addr learned my addr 172.21.15.50:0/1268446669 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.433+0000 7f43e3e00640 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe32990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 -- 172.21.15.50:0/1268446669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 msgr2=0x55e1cfe47f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 -- 172.21.15.50:0/1268446669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 msgr2=0x55e1cfe32990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe32990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 -- 172.21.15.50:0/1268446669 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cfe417a0 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43eaa00640 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43dc014040 tx=0x7f43dc0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65112d5f8130ac05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43daa00640 1 -- 172.21.15.50:0/1268446669 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc030070 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43daa00640 1 -- 172.21.15.50:0/1268446669 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc01f040 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.434+0000 7f43daa00640 1 -- 172.21.15.50:0/1268446669 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc01e040 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 -- 172.21.15.50:0/1268446669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 msgr2=0x55e1cfe32450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43dc014040 tx=0x7f43dc0051d0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 -- 172.21.15.50:0/1268446669 shutdown_connections 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe32990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 --2- 172.21.15.50:0/1268446669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe32450 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.435+0000 7f43f862d740 1 -- 172.21.15.50:0/1268446669 >> 172.21.15.50:0/1268446669 conn(0x55e1cfa89300 msgr2=0x55e1cfe37080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.436+0000 7f43f862d740 1 -- 172.21.15.50:0/1268446669 shutdown_connections 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.436+0000 7f43f862d740 1 -- 172.21.15.50:0/1268446669 wait complete. 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.436+0000 7f43f862d740 1 -- start start 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.436+0000 7f43f862d740 1 Processor -- start 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe45ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1abb10 con 0x55e1cfa951b0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1acf80 con 0x55e1cfa744d0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ac1e0 con 0x55e1cfa6c2f0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43f862d740 10 client.0 _ll_register_callbacks cb 0x7f43e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe45ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56300/0 (socket says 172.21.15.50:56300) 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe45ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58826/0 (socket says 172.21.15.50:58826) 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43eb400640 1 -- 172.21.15.50:0/2708742123 learned_addr learned my addr 172.21.15.50:0/2708742123 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.437+0000 7f43e3e00640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 -- 172.21.15.50:0/2708742123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 msgr2=0x55e1cfe45ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe45ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 -- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 msgr2=0x55e1cfe460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 -- 172.21.15.50:0/2708742123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cfe475d0 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43eb400640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f43e4030040 tx=0x7f43e4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46b8b3e8f5af801 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4030c10 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f43e4030db0 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc01f040 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43e406d050 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43f862d740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.438+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1cfe488d0 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc01f040 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43e4051070 con 0x55e1cfa951b0 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 20 client.15303 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa291c0 0x55e1cfeb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 4 client.15303 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.439+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1cfda2fd0 con 0x55e1cfa291c0 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.440+0000 7f43eaa00640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa291c0 0x55e1cfeb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.440+0000 7f43eaa00640 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa291c0 0x55e1cfeb8fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347e140930b36064 server_cookie=3ef048fba6ced0b4 in_seq=0 out_seq=0 2024-05-10T21:09:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.440+0000 7f43cfe00640 10 client.15303 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.448+0000 7f43cfe00640 1 -- 172.21.15.50:0/2708742123 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1cfda2fd0 con 0x55e1cfa291c0 2024-05-10T21:09:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.448+0000 7f43cfe00640 10 client.15303 handle_command_reply: tid=0 2024-05-10T21:09:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.448+0000 7f43f862d740 1 client.15303 shutdown 2024-05-10T21:09:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 msgr2=0x55e1cfe47010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f43e4030040 tx=0x7f43e4009c40 comp rx=0 tx=0).stop 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 shutdown_connections 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e1cfa291c0 0x55e1cfeb8fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1cfa951b0 0x55e1cfe47010 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.450+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e1cfa744d0 0x55e1cfe460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.450+0000 7f43f862d740 1 --2- 172.21.15.50:0/2708742123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1cfa6c2f0 0x55e1cfe45ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.450+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 >> 172.21.15.50:0/2708742123 conn(0x55e1cfa89300 msgr2=0x55e1cfe32340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.449+0000 7f43cfe00640 0 client.15303 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.450+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 shutdown_connections 2024-05-10T21:09:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.450+0000 7f43f862d740 1 -- 172.21.15.50:0/2708742123 wait complete. 2024-05-10T21:09:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.451+0000 7f43f862d740 20 client.15303 trim_cache size 0 max 16384 2024-05-10T21:09:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:09:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:09:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:09:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.451+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 msgr2=0x7f43d00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.451+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 0x7f43d00b4cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005e30 tx=0x7f43e4031af0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.451+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 msgr2=0x7f43ec19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.451+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e9f0 tx=0x7f43d400eeb0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 shutdown_connections 2024-05-10T21:09:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f43d00b27e0 0x7f43d00b4cf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43ec109a00 0x7f43ec1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43ec1070a0 0x7f43ec19cb00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 --2- 172.21.15.50:0/465253501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43ec104740 0x7f43ec19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 >> 172.21.15.50:0/465253501 conn(0x7f43ec0fdd40 msgr2=0x7f43ec102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 shutdown_connections 2024-05-10T21:09:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:40.452+0000 7f43f1600640 1 -- 172.21.15.50:0/465253501 wait complete. 2024-05-10T21:09:41.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:40 smithi092 ceph-mon[34415]: pgmap v484: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:09:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:40 smithi142 ceph-mon[34846]: pgmap v484: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:09:41.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:40 smithi050 ceph-mon[29546]: pgmap v484: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:09:41.574 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:42.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:41 smithi050 ceph-mon[29546]: pgmap v485: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T21:09:42.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:41 smithi092 ceph-mon[34415]: pgmap v485: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T21:09:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:41 smithi142 ceph-mon[34846]: pgmap v485: 97 pgs: 97 active+clean; 661 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 -- 172.21.15.50:0/4148631449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 msgr2=0x7f577c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 --2- 172.21.15.50:0/4148631449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c109330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5764009990 tx=0x7f5764009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 -- 172.21.15.50:0/4148631449 shutdown_connections 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 --2- 172.21.15.50:0/4148631449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 --2- 172.21.15.50:0/4148631449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 --2- 172.21.15.50:0/4148631449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c109330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 -- 172.21.15.50:0/4148631449 >> 172.21.15.50:0/4148631449 conn(0x7f577c100040 msgr2=0x7f577c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 -- 172.21.15.50:0/4148631449 shutdown_connections 2024-05-10T21:09:42.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.737+0000 7f5782200700 1 -- 172.21.15.50:0/4148631449 wait complete. 2024-05-10T21:09:42.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.738+0000 7f5782200700 1 Processor -- start 2024-05-10T21:09:42.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.738+0000 7f5782200700 1 -- start start 2024-05-10T21:09:42.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:42.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:42.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:42.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1144d0 con 0x7f577c109870 2024-05-10T21:09:42.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f577c114350 con 0x7f577c108f20 2024-05-10T21:09:42.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5782200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f577c114650 con 0x7f577c1045f0 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f577be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f577be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56318/0 (socket says 172.21.15.50:56318) 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f5772a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f577be00700 1 -- 172.21.15.50:0/860830220 learned_addr learned my addr 172.21.15.50:0/860830220 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.739+0000 7f577b400700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 -- 172.21.15.50:0/860830220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 msgr2=0x7f577c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 -- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 msgr2=0x7f577c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:42.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 -- 172.21.15.50:0/860830220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c19fdf0 con 0x7f577c108f20 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f577be00700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f577b400700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.740+0000 7f5772a00700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c19b4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.741+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764026070 con 0x7f577c108f20 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.741+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576400b610 con 0x7f577c108f20 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.741+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1a0080 con 0x7f577c108f20 2024-05-10T21:09:42.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.741+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764021dd0 con 0x7f577c108f20 2024-05-10T21:09:42.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.741+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a7a00 con 0x7f577c108f20 2024-05-10T21:09:42.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.742+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c105aa0 con 0x7f577c108f20 2024-05-10T21:09:42.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.745+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5764029070 con 0x7f577c108f20 2024-05-10T21:09:42.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.746+0000 7f5778c00700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 0x7f575c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.746+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f57640e34d0 con 0x7f577c108f20 2024-05-10T21:09:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.746+0000 7f577b400700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 0x7f575c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:42.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.747+0000 7f577b400700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 0x7f575c0b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f576c011490 tx=0x7f576c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:42.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:42.748+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5764030020 con 0x7f577c108f20 2024-05-10T21:09:43.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.062+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577c0512c0 con 0x7f577c108f20 2024-05-10T21:09:43.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.063+0000 7f5778c00700 1 -- 172.21.15.50:0/860830220 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f5764013070 con 0x7f577c108f20 2024-05-10T21:09:43.063 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:09:43.063 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:09:43.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 msgr2=0x7f575c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:43.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 0x7f575c0b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f576c011490 tx=0x7f576c009260 comp rx=0 tx=0).stop 2024-05-10T21:09:43.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 msgr2=0x7f577c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:43.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c19b4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764009580 comp rx=0 tx=0).stop 2024-05-10T21:09:43.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 shutdown_connections 2024-05-10T21:09:43.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f575c0b2900 0x7f575c0b4e10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:43.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f577c109870 0x7f577c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c1045f0 0x7f577c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 --2- 172.21.15.50:0/860830220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f577c108f20 0x7f577c19b4c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 >> 172.21.15.50:0/860830220 conn(0x7f577c100040 msgr2=0x7f577c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 shutdown_connections 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:43.067+0000 7f5782200700 1 -- 172.21.15.50:0/860830220 wait complete. 2024-05-10T21:09:43.068 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:09:43.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:43 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/860830220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:43 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/860830220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:43.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:43 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/860830220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:44.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:44 smithi050 ceph-mon[29546]: pgmap v486: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:09:44.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 -- 172.21.15.50:0/4115739474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8108800 msgr2=0x7ff9b8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 --2- 172.21.15.50:0/4115739474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8108800 0x7ff9b8108c10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0098f0 tx=0x7ff9ac009c00 comp rx=0 tx=0).stop 2024-05-10T21:09:44.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 -- 172.21.15.50:0/4115739474 shutdown_connections 2024-05-10T21:09:44.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 --2- 172.21.15.50:0/4115739474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8108800 0x7ff9b8108c10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 --2- 172.21.15.50:0/4115739474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8103120 0x7ff9b810f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 --2- 172.21.15.50:0/4115739474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.313+0000 7ff9be800700 1 -- 172.21.15.50:0/4115739474 >> 172.21.15.50:0/4115739474 conn(0x7ff9b80783b0 msgr2=0x7ff9b80787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.314+0000 7ff9be800700 1 -- 172.21.15.50:0/4115739474 shutdown_connections 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.314+0000 7ff9be800700 1 -- 172.21.15.50:0/4115739474 wait complete. 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 Processor -- start 2024-05-10T21:09:44.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 -- start start 2024-05-10T21:09:44.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8108800 0x7ff9b8104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81131b0 con 0x7ff9b81027d0 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8113030 con 0x7ff9b8103120 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.315+0000 7ff9be800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8113330 con 0x7ff9b8108800 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58026/0 (socket says 172.21.15.50:58026) 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 -- 172.21.15.50:0/2743934365 learned_addr learned my addr 172.21.15.50:0/2743934365 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52820/0 (socket says 172.21.15.50:52820) 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8108800 0x7ff9b8104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 -- 172.21.15.50:0/2743934365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8108800 msgr2=0x7ff9b8104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8108800 0x7ff9b8104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 -- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 msgr2=0x7ff9b81043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bca00700 1 -- 172.21.15.50:0/2743934365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b81a9030 con 0x7ff9b8103120 2024-05-10T21:09:44.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.316+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.323+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400e970 tx=0x7ff9a400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.323+0000 7ff9b6000700 1 -- 172.21.15.50:0/2743934365 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4004a30 con 0x7ff9b8103120 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.323+0000 7ff9b6000700 1 -- 172.21.15.50:0/2743934365 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a400cc90 con 0x7ff9b8103120 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.323+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b81a9320 con 0x7ff9b8103120 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.324+0000 7ff9b6000700 1 -- 172.21.15.50:0/2743934365 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4005610 con 0x7ff9b8103120 2024-05-10T21:09:44.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.324+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b81a98b0 con 0x7ff9b8103120 2024-05-10T21:09:44.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.327+0000 7ff9b6000700 1 -- 172.21.15.50:0/2743934365 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9a400c7e0 con 0x7ff9b8103120 2024-05-10T21:09:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.328+0000 7ff9ce8fab80 1 Processor -- start 2024-05-10T21:09:44.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 -- start start 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 0x555e39496540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39490c40 0x555e39491050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555e393b2a20 con 0x555e39490c40 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555e39350760 con 0x555e39496a80 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555e393ae9e0 con 0x555e39312e40 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58038/0 (socket says 172.21.15.50:58038) 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9bd400700 1 -- 172.21.15.50:0/2683335331 learned_addr learned my addr 172.21.15.50:0/2683335331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.329+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39490c40 0x555e39491050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 0x555e39496540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 -- 172.21.15.50:0/2683335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 msgr2=0x555e39496540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9b6000700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 0x7ff9a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 0x555e39496540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 -- 172.21.15.50:0/2683335331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39490c40 msgr2=0x555e39491050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39490c40 0x555e39491050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 -- 172.21.15.50:0/2683335331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e392ff8d0 con 0x555e39496a80 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 0x555e39496540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9b6000700 1 -- 172.21.15.50:0/2743934365 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff9a40e24d0 con 0x7ff9b8103120 2024-05-10T21:09:44.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.330+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0098f0 tx=0x7ff9ac005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a556ee6ac7242112 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.331+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 0x7ff9a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.331+0000 7ff9aaa00700 1 -- 172.21.15.50:0/2683335331 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00f760 con 0x555e39496a80 2024-05-10T21:09:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.331+0000 7ff9aaa00700 1 -- 172.21.15.50:0/2683335331 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ac00f900 con 0x555e39496a80 2024-05-10T21:09:44.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.331+0000 7ff9aaa00700 1 -- 172.21.15.50:0/2683335331 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac004d00 con 0x555e39496a80 2024-05-10T21:09:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2683335331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 msgr2=0x555e39490700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 0x7ff9a00b4e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x555e394919b0 tx=0x7ff9b0004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0098f0 tx=0x7ff9ac005dc0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2683335331 shutdown_connections 2024-05-10T21:09:44.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39490c40 0x555e39491050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39312e40 0x555e39496540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2683335331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e39490700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2683335331 >> 172.21.15.50:0/2683335331 conn(0x555e39357f70 msgr2=0x555e3948b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2683335331 shutdown_connections 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.332+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2683335331 wait complete. 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.333+0000 7ff9ce8fab80 1 -- start start 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.333+0000 7ff9ce8fab80 1 Processor -- start 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.333+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.333+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e3949a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.333+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e3949ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555e3949d490 con 0x555e39312e40 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555e39318b20 con 0x555e39496a80 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555e3949d610 con 0x555e39490c40 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9ce8fab80 10 client.0 _ll_register_callbacks cb 0x555e39359c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e3949a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52842/0 (socket says 172.21.15.50:52842) 2024-05-10T21:09:44.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 -- 172.21.15.50:0/2776113431 learned_addr learned my addr 172.21.15.50:0/2776113431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e3949ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 -- 172.21.15.50:0/2776113431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 msgr2=0x555e3949a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e3949a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 -- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 msgr2=0x555e3949ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e3949ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.334+0000 7ff9bd400700 1 -- 172.21.15.50:0/2776113431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e39378420 con 0x555e39312e40 2024-05-10T21:09:44.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009810 tx=0x7ff9ac005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bf3e2372177a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff99f400700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac030070 con 0x555e39312e40 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ac00f900 con 0x555e39312e40 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff9ac003880 con 0x555e39312e40 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac03c050 con 0x555e39312e40 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff9ce8fab80 10 client.15321 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.335+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e394b08b0 con 0x555e39312e40 2024-05-10T21:09:44.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ac00f900 con 0x555e39312e40 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff9ac00bc90 con 0x555e39312e40 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 20 client.15321 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e394b2d80 0x555e394b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 4 client.15321 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:44.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e3945d1c0 con 0x555e394b2d80 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.336+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e394b2d80 0x555e394b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.337+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e394b2d80 0x555e394b51a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9d3386f6d1c8af server_cookie=72161b8a4a7f1cc in_seq=0 out_seq=0 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.337+0000 7ff99f400700 10 client.15321 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff99f400700 1 -- 172.21.15.50:0/2776113431 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e3945d1c0 con 0x555e394b2d80 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff99f400700 10 client.15321 handle_command_reply: tid=0 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 client.15321 shutdown 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 msgr2=0x555e3949a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009810 tx=0x7ff9ac005280 comp rx=0 tx=0).stop 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 shutdown_connections 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e394b2d80 0x555e394b51a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39496a80 0x555e3949ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e3949a1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2776113431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e3949a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff99f400700 0 client.15321 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 >> 172.21.15.50:0/2776113431 conn(0x555e39357f70 msgr2=0x555e3948f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 shutdown_connections 2024-05-10T21:09:44.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.338+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2776113431 wait complete. 2024-05-10T21:09:44.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.339+0000 7ff9ce8fab80 20 client.15321 trim_cache size 0 max 16384 2024-05-10T21:09:44.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 Processor -- start 2024-05-10T21:09:44.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 -- start start 2024-05-10T21:09:44.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e39490700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 0x555e394af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81aef90 con 0x555e39312e40 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81ac820 con 0x555e39490c40 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.360+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b819e700 con 0x555e39496a80 2024-05-10T21:09:44.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e39490700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 0x555e394af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49228/0 (socket says 172.21.15.50:49228) 2024-05-10T21:09:44.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bde00700 1 -- 172.21.15.50:0/2081992924 learned_addr learned my addr 172.21.15.50:0/2081992924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:44.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 -- 172.21.15.50:0/2081992924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 msgr2=0x555e39495bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 -- 172.21.15.50:0/2081992924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 msgr2=0x555e39490700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e39490700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bca00700 1 -- 172.21.15.50:0/2081992924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3948c410 con 0x555e39490c40 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bde00700 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.361+0000 7ff9bd400700 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e39490700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.362+0000 7ff9bca00700 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 0x555e394af1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40a9a50 tx=0x7ff9a40cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b431f9d1a95ce2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.362+0000 7ff9aa000700 1 -- 172.21.15.50:0/2081992924 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40a5980 con 0x555e39490c40 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.362+0000 7ff9aa000700 1 -- 172.21.15.50:0/2081992924 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a400cad0 con 0x555e39490c40 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.362+0000 7ff9aa000700 1 -- 172.21.15.50:0/2081992924 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40a5250 con 0x555e39490c40 2024-05-10T21:09:44.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2081992924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 msgr2=0x555e394af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 0x555e394af1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40a9a50 tx=0x7ff9a40cc040 comp rx=0 tx=0).stop 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2081992924 shutdown_connections 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39496a80 0x555e39495bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39312e40 0x555e39490700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/2081992924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39490c40 0x555e394af1a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2081992924 >> 172.21.15.50:0/2081992924 conn(0x555e39357f70 msgr2=0x555e394852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2081992924 shutdown_connections 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.363+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/2081992924 wait complete. 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 -- start start 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 Processor -- start 2024-05-10T21:09:44.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e394b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 0x555e394b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81aad80 con 0x555e39496a80 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81ac150 con 0x555e39312e40 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81ab3b0 con 0x555e39490c40 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.364+0000 7ff9ce8fab80 10 client.0 _ll_register_callbacks cb 0x555e39359c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bde00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 0x555e394b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58066/0 (socket says 172.21.15.50:58066) 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bde00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 0x555e394b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52854/0 (socket says 172.21.15.50:52854) 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 -- 172.21.15.50:0/3367951907 learned_addr learned my addr 172.21.15.50:0/3367951907 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bca00700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e394b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 -- 172.21.15.50:0/3367951907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 msgr2=0x555e394b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e394b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 -- 172.21.15.50:0/3367951907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 msgr2=0x555e394b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 0x555e394b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 -- 172.21.15.50:0/3367951907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3951c4c0 con 0x555e39312e40 2024-05-10T21:09:44.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff9bd400700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x555e39499a20 tx=0x7ff9ac00fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e0972d144b8a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.365+0000 7ff99f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac030040 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff9ac003680 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e393c0400 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac0053a0 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff9ce8fab80 10 client.25445 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.366+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e3951d730 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e393c0400 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff9ac00bba0 con 0x555e39312e40 2024-05-10T21:09:44.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 10 client.25445 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 20 client.25445 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e3951dd20 0x555e39520140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 4 client.25445 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e393ba4a0 con 0x555e3951dd20 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.367+0000 7ff9bca00700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e3951dd20 0x555e39520140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9bca00700 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e3951dd20 0x555e39520140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f08bbf490b62c50 server_cookie=399666749be83b31 in_seq=0 out_seq=0 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff99f400700 10 client.25445 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff99f400700 1 -- 172.21.15.50:0/3367951907 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e393ba4a0 con 0x555e3951dd20 2024-05-10T21:09:44.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff99f400700 10 client.25445 handle_command_reply: tid=0 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 client.25445 shutdown 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 msgr2=0x555e394b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x555e39499a20 tx=0x7ff9ac00fb00 comp rx=0 tx=0).stop 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 shutdown_connections 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x555e3951dd20 0x555e39520140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e39312e40 0x555e394b6000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555e39496a80 0x555e394b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff99f400700 0 client.25445 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 --2- 172.21.15.50:0/3367951907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555e39490c40 0x555e394b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 >> 172.21.15.50:0/3367951907 conn(0x555e39357f70 msgr2=0x555e394852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 shutdown_connections 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.368+0000 7ff9ce8fab80 1 -- 172.21.15.50:0/3367951907 wait complete. 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9ce8fab80 20 client.25445 trim_cache size 0 max 16384 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:09:44.376 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:09:44.377 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:44.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 msgr2=0x7ff9a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 0x7ff9a00b4e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x555e394919b0 tx=0x7ff9b0004ad0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 msgr2=0x7ff9b81048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400e970 tx=0x7ff9a400ec80 comp rx=0 tx=0).stop 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 shutdown_connections 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff9a00b2900 0x7ff9a00b4e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9b8108800 0x7ff9b8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b81027d0 0x7ff9b81043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 --2- 172.21.15.50:0/2743934365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9b8103120 0x7ff9b81048e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:44.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.369+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 >> 172.21.15.50:0/2743934365 conn(0x7ff9b80783b0 msgr2=0x7ff9b8100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:44.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.370+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 shutdown_connections 2024-05-10T21:09:44.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:44.370+0000 7ff9be800700 1 -- 172.21.15.50:0/2743934365 wait complete. 2024-05-10T21:09:44.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:44 smithi142 ceph-mon[34846]: pgmap v486: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:09:44.618 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:44 smithi092 ceph-mon[34415]: pgmap v486: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-10T21:09:44.808 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 60 --expiration 81.35 --await-for 42.0' 2024-05-10T21:09:44.808 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db / --timeout 60 --expiration 81.35 --await-for 42.0 2024-05-10T21:09:45.517 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:09:45.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:09:45.669 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:47.088 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:46 smithi050 ceph-mon[29546]: pgmap v487: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:47.089 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:46 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:09:47.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:46 smithi092 ceph-mon[34415]: pgmap v487: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:47.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:46 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:09:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:46 smithi142 ceph-mon[34846]: pgmap v487: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:46 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:09:47.179 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:09:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:09:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:47 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:09:48.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:09:48.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:09:48.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:47 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:09:48.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:09:48.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:09:48.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:47 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:09:48.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.639+0000 7f6322200700 1 -- 172.21.15.50:0/632136820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c106ec0 msgr2=0x7f631c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.639+0000 7f6322200700 1 --2- 172.21.15.50:0/632136820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c106ec0 0x7f631c113350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6314009f90 tx=0x7f631400b330 comp rx=0 tx=0).stop 2024-05-10T21:09:48.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 -- 172.21.15.50:0/632136820 shutdown_connections 2024-05-10T21:09:48.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 --2- 172.21.15.50:0/632136820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c106ec0 0x7f631c113350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 --2- 172.21.15.50:0/632136820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c10c5b0 0x7f631c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 --2- 172.21.15.50:0/632136820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106570 0x7f631c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 -- 172.21.15.50:0/632136820 >> 172.21.15.50:0/632136820 conn(0x7f631c102050 msgr2=0x7f631c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 -- 172.21.15.50:0/632136820 shutdown_connections 2024-05-10T21:09:48.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.641+0000 7f6322200700 1 -- 172.21.15.50:0/632136820 wait complete. 2024-05-10T21:09:48.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 Processor -- start 2024-05-10T21:09:48.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 -- start start 2024-05-10T21:09:48.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c106570 0x7f631c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106ec0 0x7f631c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f631c116750 con 0x7f631c106570 2024-05-10T21:09:48.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1165d0 con 0x7f631c10c5b0 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.643+0000 7f6322200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1168d0 con 0x7f631c106ec0 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c106570 0x7f631c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58076/0 (socket says 172.21.15.50:58076) 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 -- 172.21.15.50:0/3703495617 learned_addr learned my addr 172.21.15.50:0/3703495617 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:48.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631aa00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106ec0 0x7f631c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 -- 172.21.15.50:0/3703495617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106ec0 msgr2=0x7f631c19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106ec0 0x7f631c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 -- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c106570 msgr2=0x7f631c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c106570 0x7f631c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.644+0000 7f631be00700 1 -- 172.21.15.50:0/3703495617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1a1ed0 con 0x7f631c10c5b0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f631be00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f631400b6c0 tx=0x7f631400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f6313e00700 1 -- 172.21.15.50:0/3703495617 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314021070 con 0x7f631c10c5b0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f6313e00700 1 -- 172.21.15.50:0/3703495617 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6314007ac0 con 0x7f631c10c5b0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1a2160 con 0x7f631c10c5b0 2024-05-10T21:09:48.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f6313e00700 1 -- 172.21.15.50:0/3703495617 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140052a0 con 0x7f631c10c5b0 2024-05-10T21:09:48.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.645+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1a9ae0 con 0x7f631c10c5b0 2024-05-10T21:09:48.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.649+0000 7f6313e00700 1 -- 172.21.15.50:0/3703495617 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6314024070 con 0x7f631c10c5b0 2024-05-10T21:09:48.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.651+0000 7f6330c8ab80 1 Processor -- start 2024-05-10T21:09:48.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.651+0000 7f6313e00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 0x7f63000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.651+0000 7f6330c8ab80 1 -- start start 2024-05-10T21:09:48.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.651+0000 7f6313e00700 1 -- 172.21.15.50:0/3703495617 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f631400f020 con 0x7f631c10c5b0 2024-05-10T21:09:48.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.651+0000 7f631aa00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 0x7f63000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 0x55e096232c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e096237470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e096154bb0 con 0x55e0962379b0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0960f28f0 con 0x55e0962331d0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e096150b70 con 0x55e0960b4fd0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f631be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f631aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e096237470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f631be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52878/0 (socket says 172.21.15.50:52878) 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f631be00700 1 -- 172.21.15.50:0/1855123852 learned_addr learned my addr 172.21.15.50:0/1855123852 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.652+0000 7f631b400700 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 0x55e096232c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631aa00700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 0x7f63000b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f631c1053d0 tx=0x7f6308009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 -- 172.21.15.50:0/1855123852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 msgr2=0x55e096237470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e096237470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 -- 172.21.15.50:0/1855123852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 msgr2=0x55e096237dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 -- 172.21.15.50:0/1855123852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0960a1a60 con 0x55e0960b4fd0 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631be00700 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:48.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.653+0000 7f631b400700 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 0x55e096232c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f631c1079e0 tx=0x7f630c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2066aebb6918d796 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.654+0000 7f6310c00700 1 -- 172.21.15.50:0/1855123852 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c010c70 con 0x55e0960b4fd0 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.654+0000 7f6310c00700 1 -- 172.21.15.50:0/1855123852 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c0044e0 con 0x55e0960b4fd0 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.654+0000 7f6310c00700 1 -- 172.21.15.50:0/1855123852 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c005220 con 0x55e0960b4fd0 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 -- 172.21.15.50:0/1855123852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 msgr2=0x55e096232c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 0x55e096232c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f631c1079e0 tx=0x7f630c00bf60 comp rx=0 tx=0).stop 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 -- 172.21.15.50:0/1855123852 shutdown_connections 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962379b0 0x55e096237dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960b4fd0 0x55e096232c90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/1855123852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e096237470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 -- 172.21.15.50:0/1855123852 >> 172.21.15.50:0/1855123852 conn(0x55e0960fa100 msgr2=0x55e09622da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 -- 172.21.15.50:0/1855123852 shutdown_connections 2024-05-10T21:09:48.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.655+0000 7f6330c8ab80 1 -- 172.21.15.50:0/1855123852 wait complete. 2024-05-10T21:09:48.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.656+0000 7f6330c8ab80 1 -- start start 2024-05-10T21:09:48.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 Processor -- start 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e09623c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e09623cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 0x55e09623d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e09623f900 con 0x55e0960b4fd0 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e0960bacb0 con 0x55e0962331d0 2024-05-10T21:09:48.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.657+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e09623fa80 con 0x55e0962379b0 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f6330c8ab80 10 client.0 _ll_register_callbacks cb 0x55e0960fbe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 0x55e09623d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e09623c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 0x55e09623d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49264/0 (socket says 172.21.15.50:49264) 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e09623cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631be00700 1 -- 172.21.15.50:0/170256699 learned_addr learned my addr 172.21.15.50:0/170256699 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631b400700 1 -- 172.21.15.50:0/170256699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 msgr2=0x55e09623d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631b400700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 0x55e09623d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.658+0000 7f631b400700 1 -- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 msgr2=0x55e09623cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f631b400700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e09623cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f631b400700 1 -- 172.21.15.50:0/170256699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e09611a5b0 con 0x55e0960b4fd0 2024-05-10T21:09:48.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f631aa00700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e09623cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:48.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f631b400700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e09623c690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f630c0059d0 tx=0x7f630c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7084cd477b70a8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f6305600700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c02a060 con 0x55e0960b4fd0 2024-05-10T21:09:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c0044e0 con 0x55e0960b4fd0 2024-05-10T21:09:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.659+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f630c010d80 con 0x55e0960b4fd0 2024-05-10T21:09:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.660+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f630c004db0 con 0x55e0960b4fd0 2024-05-10T21:09:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.660+0000 7f6330c8ab80 10 client.15339 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.660+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e09624a4d0 con 0x55e0960b4fd0 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.660+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c0044e0 con 0x55e0960b4fd0 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.660+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f630c0116e0 con 0x55e0960b4fd0 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 10 client.15339 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 20 client.15339 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e09624ca00 0x55e09624ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 4 client.15339 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:48.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0961ff350 con 0x55e09624ca00 2024-05-10T21:09:48.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.661+0000 7f631aa00700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e09624ca00 0x55e09624ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.662+0000 7f6305600700 10 client.15339 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:48.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.662+0000 7f631aa00700 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e09624ca00 0x55e09624ee20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef522c703aba9c5 server_cookie=473191a671dd9d41 in_seq=0 out_seq=0 2024-05-10T21:09:48.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6305600700 1 -- 172.21.15.50:0/170256699 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0961ff350 con 0x55e09624ca00 2024-05-10T21:09:48.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6305600700 10 client.15339 handle_command_reply: tid=0 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 client.15339 shutdown 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 msgr2=0x55e09623c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e09623c690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f630c0059d0 tx=0x7f630c004ac0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 shutdown_connections 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e09624ca00 0x55e09624ee20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962379b0 0x55e09623d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e09623c690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/170256699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0962331d0 0x55e09623cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6305600700 0 client.15339 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:48.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.663+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 >> 172.21.15.50:0/170256699 conn(0x55e0960fa100 msgr2=0x55e0962319e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.664+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 shutdown_connections 2024-05-10T21:09:48.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.664+0000 7f6330c8ab80 1 -- 172.21.15.50:0/170256699 wait complete. 2024-05-10T21:09:48.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.664+0000 7f6330c8ab80 20 client.15339 trim_cache size 0 max 16384 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.694+0000 7f6330c8ab80 1 Processor -- start 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.694+0000 7f6330c8ab80 1 -- start start 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 0x55e096232410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960b4fd0 0x55e09623cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b3510 con 0x55e0962331d0 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b0da0 con 0x55e0960b4fd0 2024-05-10T21:09:48.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1aec90 con 0x55e0960fa100 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f631be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f631b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 0x55e096232410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f631be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52898/0 (socket says 172.21.15.50:52898) 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f631be00700 1 -- 172.21.15.50:0/698540528 learned_addr learned my addr 172.21.15.50:0/698540528 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.695+0000 7f631aa00700 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960b4fd0 0x55e09623cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 -- 172.21.15.50:0/698540528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960b4fd0 msgr2=0x55e09623cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960b4fd0 0x55e09623cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 -- 172.21.15.50:0/698540528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 msgr2=0x55e096238dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 -- 172.21.15.50:0/698540528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e09623d750 con 0x55e0960fa100 2024-05-10T21:09:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631be00700 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f631b400700 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 0x55e096232410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f630c00bbd0 tx=0x7f630c00b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef079423f90d895 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.696+0000 7f6307e00700 1 -- 172.21.15.50:0/698540528 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c035070 con 0x55e0960fa100 2024-05-10T21:09:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.697+0000 7f6307e00700 1 -- 172.21.15.50:0/698540528 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c004be0 con 0x55e0960fa100 2024-05-10T21:09:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.697+0000 7f6307e00700 1 -- 172.21.15.50:0/698540528 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c02a070 con 0x55e0960fa100 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- 172.21.15.50:0/698540528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 msgr2=0x55e096232410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 0x55e096232410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f630c00bbd0 tx=0x7f630c00b540 comp rx=0 tx=0).stop 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- 172.21.15.50:0/698540528 shutdown_connections 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0962331d0 0x55e096238dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0960fa100 0x55e096232410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/698540528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960b4fd0 0x55e09623cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- 172.21.15.50:0/698540528 >> 172.21.15.50:0/698540528 conn(0x55e0962270e0 msgr2=0x55e096239620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- 172.21.15.50:0/698540528 shutdown_connections 2024-05-10T21:09:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- 172.21.15.50:0/698540528 wait complete. 2024-05-10T21:09:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.698+0000 7f6330c8ab80 1 -- start start 2024-05-10T21:09:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 Processor -- start 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 0x55e0962505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962331d0 0x55e096250b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1af2e0 con 0x55e0960b4fd0 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b06d0 con 0x55e0960fa100 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1af930 con 0x55e0962331d0 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.699+0000 7f6330c8ab80 10 client.0 _ll_register_callbacks cb 0x55e09622e060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 0x55e0962505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52914/0 (socket says 172.21.15.50:52914) 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631b400700 1 -- 172.21.15.50:0/552898800 learned_addr learned my addr 172.21.15.50:0/552898800 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631be00700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962331d0 0x55e096250b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 -- 172.21.15.50:0/552898800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962331d0 msgr2=0x55e096250b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962331d0 0x55e096250b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 -- 172.21.15.50:0/552898800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 msgr2=0x55e096252240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631aa00700 1 -- 172.21.15.50:0/552898800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e096251180 con 0x55e0960fa100 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.700+0000 7f631b400700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f631aa00700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 0x55e0962505d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6308009940 tx=0x7f6308004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebed1bb887bd30f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6305600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308004a30 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6308004bd0 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c004be0 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6308004dd0 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6330c8ab80 10 client.25463 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.701+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0962b7d80 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c004be0 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f630800f040 con 0x55e0960fa100 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 10 client.25463 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 10 client.25463 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 20 client.25463 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e0962379b0 0x55e0962ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 4 client.25463 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.35, "await_for": 42.0}] 2024-05-10T21:09:48.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.702+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.35, "await_for": 42.0}) -- 0x55e096162590 con 0x55e0962379b0 2024-05-10T21:09:48.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.703+0000 7f631b400700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e0962379b0 0x55e0962ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:48.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.703+0000 7f631b400700 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e0962379b0 0x55e0962ba590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7f3116fee5c223 server_cookie=6c20841164fcd8b7 in_seq=0 out_seq=0 2024-05-10T21:09:48.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.703+0000 7f6305600700 10 client.25463 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:48.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6305600700 1 -- 172.21.15.50:0/552898800 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e096162590 con 0x55e0962379b0 2024-05-10T21:09:48.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6305600700 10 client.25463 handle_command_reply: tid=0 2024-05-10T21:09:48.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 client.25463 shutdown 2024-05-10T21:09:48.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 msgr2=0x55e0962505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 0x55e0962505d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6308009940 tx=0x7f6308004750 comp rx=0 tx=0).stop 2024-05-10T21:09:48.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 shutdown_connections 2024-05-10T21:09:48.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e0962379b0 0x55e0962ba590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e0962331d0 0x55e096250b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0960b4fd0 0x55e096252240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 --2- 172.21.15.50:0/552898800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0960fa100 0x55e0962505d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 >> 172.21.15.50:0/552898800 conn(0x55e0962270e0 msgr2=0x55e096232170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6305600700 0 client.25463 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 shutdown_connections 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.712+0000 7f6330c8ab80 1 -- 172.21.15.50:0/552898800 wait complete. 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6330c8ab80 20 client.25463 trim_cache size 0 max 16384 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 25, 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:09:48.713 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 2, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "b63a33a6": { 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 2, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 81.3, 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:09:48.714 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:09:48.715 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 msgr2=0x7f63000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 0x7f63000b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f631c1053d0 tx=0x7f6308009310 comp rx=0 tx=0).stop 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 msgr2=0x7f631c1a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f631400b6c0 tx=0x7f631400cd70 comp rx=0 tx=0).stop 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 shutdown_connections 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f63000b2880 0x7f63000b4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f631c10c5b0 0x7f631c1a1830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631c106570 0x7f631c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.713+0000 7f6322200700 1 --2- 172.21.15.50:0/3703495617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f631c106ec0 0x7f631c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:48.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.714+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 >> 172.21.15.50:0/3703495617 conn(0x7f631c102050 msgr2=0x7f631c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:48.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.714+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 shutdown_connections 2024-05-10T21:09:48.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:48.714+0000 7f6322200700 1 -- 172.21.15.50:0/3703495617 wait complete. 2024-05-10T21:09:48.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:48 smithi050 ceph-mon[29546]: pgmap v488: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T21:09:49.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:48 smithi092 ceph-mon[34415]: pgmap v488: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T21:09:49.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:48 smithi142 ceph-mon[34846]: pgmap v488: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T21:09:50.136 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b63a33a6', quiesce took 10.6 seconds. Will release after: 21.35 2024-05-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.616+0000 7fb1da800700 1 -- 172.21.15.50:0/4230484010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d4105c00 msgr2=0x7fb1d410e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.616+0000 7fb1da800700 1 --2- 172.21.15.50:0/4230484010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d4105c00 0x7fb1d410e150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009990 tx=0x7fb1c0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 -- 172.21.15.50:0/4230484010 shutdown_connections 2024-05-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 --2- 172.21.15.50:0/4230484010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 0x7fb1d4110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 --2- 172.21.15.50:0/4230484010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d41052b0 0x7fb1d41056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 --2- 172.21.15.50:0/4230484010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d4105c00 0x7fb1d410e150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 -- 172.21.15.50:0/4230484010 >> 172.21.15.50:0/4230484010 conn(0x7fb1d4078af0 msgr2=0x7fb1d4078f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 -- 172.21.15.50:0/4230484010 shutdown_connections 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.617+0000 7fb1da800700 1 -- 172.21.15.50:0/4230484010 wait complete. 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.618+0000 7fb1da800700 1 Processor -- start 2024-05-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.618+0000 7fb1da800700 1 -- start start 2024-05-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d4105c00 0x7fb1d41034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 0x7fb1d4103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4114280 con 0x7fb1d41052b0 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4114100 con 0x7fb1d410e720 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1da800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4114400 con 0x7fb1d4105c00 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1d3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1d3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52930/0 (socket says 172.21.15.50:52930) 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1d3400700 1 -- 172.21.15.50:0/2469971703 learned_addr learned my addr 172.21.15.50:0/2469971703 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1d2a00700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d4105c00 0x7fb1d41034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.619+0000 7fb1d3e00700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 0x7fb1d4103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 -- 172.21.15.50:0/2469971703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d4105c00 msgr2=0x7fb1d41034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d4105c00 0x7fb1d41034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 -- 172.21.15.50:0/2469971703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 msgr2=0x7fb1d4103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 0x7fb1d4103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 -- 172.21.15.50:0/2469971703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d41a9070 con 0x7fb1d41052b0 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1d3400700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9a0 tx=0x7fb1c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.620+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400ca10 con 0x7fb1d41052b0 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.621+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c400cbb0 con 0x7fb1d41052b0 2024-05-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.621+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4018680 con 0x7fb1d41052b0 2024-05-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.621+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d41a9360 con 0x7fb1d41052b0 2024-05-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.622+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d41a9820 con 0x7fb1d41052b0 2024-05-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.623+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d40512c0 con 0x7fb1d41052b0 2024-05-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.625+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb1c4010430 con 0x7fb1d41052b0 2024-05-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.627+0000 7fb1cbe00700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 0x7fb1b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.627+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb1c40e2ed0 con 0x7fb1d41052b0 2024-05-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.627+0000 7fb1d2a00700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 0x7fb1b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.628+0000 7fb1d2a00700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 0x7fb1b80b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098e0 tx=0x7fb1c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.630+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1c4026070 con 0x7fb1d41052b0 2024-05-10T21:09:50.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:50 smithi050 ceph-mon[29546]: pgmap v489: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:50.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.955+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d402e5f0 con 0x7fb1d41052b0 2024-05-10T21:09:50.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.957+0000 7fb1cbe00700 1 -- 172.21.15.50:0/2469971703 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fb1c40aabc0 con 0x7fb1d41052b0 2024-05-10T21:09:50.957 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:09:50.957 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:09:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.960+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 msgr2=0x7fb1b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.960+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 0x7fb1b80b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098e0 tx=0x7fb1c0005960 comp rx=0 tx=0).stop 2024-05-10T21:09:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.960+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 msgr2=0x7fb1d4102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.960+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9a0 tx=0x7fb1c400ecb0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 shutdown_connections 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb1b80b2880 0x7fb1b80b4d90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d410e720 0x7fb1d4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d41052b0 0x7fb1d4102fa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 --2- 172.21.15.50:0/2469971703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1d4105c00 0x7fb1d41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 >> 172.21.15.50:0/2469971703 conn(0x7fb1d4078af0 msgr2=0x7fb1d40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:50.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 shutdown_connections 2024-05-10T21:09:50.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:50.961+0000 7fb1da800700 1 -- 172.21.15.50:0/2469971703 wait complete. 2024-05-10T21:09:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:09:51.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:50 smithi092 ceph-mon[34415]: pgmap v489: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:51.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:50 smithi142 ceph-mon[34846]: pgmap v489: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:51.934 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:09:52.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:51 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2469971703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:52.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:51 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2469971703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:52.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:51 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2469971703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:09:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:52.904 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:52 smithi050 ceph-mon[29546]: pgmap v490: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:53.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:52 smithi092 ceph-mon[34415]: pgmap v490: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:53.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:52 smithi142 ceph-mon[34846]: pgmap v490: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:09:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.693+0000 7f5d54a00640 1 -- 172.21.15.50:0/3072176930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50109820 msgr2=0x7f5d5010bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.693+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3072176930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50109820 0x7f5d5010bc40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009ca0 tx=0x7f5d4802a8c0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 -- 172.21.15.50:0/3072176930 shutdown_connections 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3072176930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50109820 0x7f5d5010bc40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3072176930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 0x7f5d501092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3072176930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50104560 0x7f5d50106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 -- 172.21.15.50:0/3072176930 >> 172.21.15.50:0/3072176930 conn(0x7f5d500fdb60 msgr2=0x7f5d50100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 -- 172.21.15.50:0/3072176930 shutdown_connections 2024-05-10T21:09:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.694+0000 7f5d54a00640 1 -- 172.21.15.50:0/3072176930 wait complete. 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.695+0000 7f5d54a00640 1 Processor -- start 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.695+0000 7f5d54a00640 1 -- start start 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.695+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.695+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 0x7f5d5019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d54a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 0x7f5d501a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d54a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010f5a0 con 0x7f5d50106ec0 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d54a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010f420 con 0x7f5d50104560 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d54a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010f720 con 0x7f5d50109820 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 0x7f5d5019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58150/0 (socket says 172.21.15.50:58150) 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 -- 172.21.15.50:0/3171285357 learned_addr learned my addr 172.21.15.50:0/3171285357 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 0x7f5d501a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 -- 172.21.15.50:0/3171285357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 msgr2=0x7f5d501a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 0x7f5d501a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 -- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 msgr2=0x7f5d5019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 0x7f5d5019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4f400640 1 -- 172.21.15.50:0/3171285357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501a1cb0 con 0x7f5d50104560 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.696+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 0x7f5d501a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d4f400640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000e9f0 tx=0x7f5d4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d4cc00640 1 -- 172.21.15.50:0/3171285357 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000ccc0 con 0x7f5d50104560 2024-05-10T21:09:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d4cc00640 1 -- 172.21.15.50:0/3171285357 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d400054d0 con 0x7f5d50104560 2024-05-10T21:09:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d501a1fa0 con 0x7f5d50104560 2024-05-10T21:09:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d4cc00640 1 -- 172.21.15.50:0/3171285357 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40010430 con 0x7f5d50104560 2024-05-10T21:09:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.697+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501a23b0 con 0x7f5d50104560 2024-05-10T21:09:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.700+0000 7f5d5cfa8740 1 Processor -- start 2024-05-10T21:09:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.700+0000 7f5d5cfa8740 1 -- start start 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce232f0 0x56471d1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56471d152550 con 0x56471ce4c1b0 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56471d0f7bc0 con 0x56471ce2b4d0 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56471d14e510 con 0x56471ce232f0 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:58164/0 (socket says 172.21.15.50:58164) 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d46a00640 1 -- 172.21.15.50:0/144657494 learned_addr learned my addr 172.21.15.50:0/144657494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.701+0000 7f5d4cc00640 1 -- 172.21.15.50:0/3171285357 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d400105d0 con 0x7f5d50104560 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 -- 172.21.15.50:0/144657494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce232f0 msgr2=0x56471d1f2c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce232f0 0x56471d1f2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 -- 172.21.15.50:0/144657494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 msgr2=0x56471d1f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d1f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 -- 172.21.15.50:0/144657494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471ce64210 con 0x56471ce2b4d0 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d1f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d46a00640 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009c90 tx=0x7f5d38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275e8ca0c34b7393 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d3fe00640 1 -- 172.21.15.50:0/144657494 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005220 con 0x56471ce2b4d0 2024-05-10T21:09:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d3fe00640 1 -- 172.21.15.50:0/144657494 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d38004c50 con 0x56471ce2b4d0 2024-05-10T21:09:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.702+0000 7f5d3fe00640 1 -- 172.21.15.50:0/144657494 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38019760 con 0x56471ce2b4d0 2024-05-10T21:09:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/144657494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 msgr2=0x56471d1f9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009c90 tx=0x7f5d38011810 comp rx=0 tx=0).stop 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d4cc00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 0x7f5d340b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/144657494 shutdown_connections 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce232f0 0x56471d1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1f9b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/144657494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d1f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d4cc00640 1 -- 172.21.15.50:0/3171285357 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d400e26c0 con 0x7f5d50104560 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/144657494 >> 172.21.15.50:0/144657494 conn(0x56471ce40300 msgr2=0x56471d1e9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 0x7f5d340b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/144657494 shutdown_connections 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.704+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/144657494 wait complete. 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.705+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 0x7f5d340b4cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x56471d1f38e0 tx=0x7f5d48031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.705+0000 7f5d5cfa8740 1 -- start start 2024-05-10T21:09:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.705+0000 7f5d5cfa8740 1 Processor -- start 2024-05-10T21:09:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56471d1fa5e0 con 0x56471ce232f0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56471d10b660 con 0x56471ce2b4d0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56471d1fa760 con 0x56471ce4c1b0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d5cfa8740 10 client.0 _ll_register_callbacks cb 0x56471d0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52974/0 (socket says 172.21.15.50:52974) 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d4f400640 1 -- 172.21.15.50:0/1205486055 learned_addr learned my addr 172.21.15.50:0/1205486055 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d46a00640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.706+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 -- 172.21.15.50:0/1205486055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 msgr2=0x56471d2036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 -- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 msgr2=0x56471d1fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 -- 172.21.15.50:0/1205486055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471cf30620 con 0x56471ce232f0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c6a0 tx=0x7f5d400a5500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae017d570fb8e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d400aa3b0 con 0x56471ce232f0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d400c4340 con 0x56471ce232f0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.707+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d38004c50 con 0x56471ce232f0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d400a9cb0 con 0x56471ce232f0 2024-05-10T21:09:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 10 client.15363 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56471d2066a0 con 0x56471ce232f0 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d38004c50 con 0x56471ce232f0 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d400c4020 con 0x56471ce232f0 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 20 client.15363 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471ce0ad80 0x56471d209d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 4 client.15363 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.708+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471d160160 con 0x56471ce0ad80 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.709+0000 7f5d46a00640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471ce0ad80 0x56471d209d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.709+0000 7f5d46a00640 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471ce0ad80 0x56471d209d90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20652bc7a2a662b2 server_cookie=bc4cf07dcd50ccc3 in_seq=0 out_seq=0 2024-05-10T21:09:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.709+0000 7f5d3cc00640 10 client.15363 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.710+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1205486055 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56471d160160 con 0x56471ce0ad80 2024-05-10T21:09:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.710+0000 7f5d3cc00640 10 client.15363 handle_command_reply: tid=0 2024-05-10T21:09:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.710+0000 7f5d5cfa8740 1 client.15363 shutdown 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.710+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 msgr2=0x56471d1fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.710+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c6a0 tx=0x7f5d400a5500 comp rx=0 tx=0).stop 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 shutdown_connections 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471ce0ad80 0x56471d209d90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce2b4d0 0x56471d1fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1205486055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce232f0 0x56471d1fc200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d3cc00640 0 client.15363 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 >> 172.21.15.50:0/1205486055 conn(0x56471ce40300 msgr2=0x56471d1e8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 shutdown_connections 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1205486055 wait complete. 2024-05-10T21:09:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.711+0000 7f5d5cfa8740 20 client.15363 trim_cache size 0 max 16384 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 Processor -- start 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 -- start start 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d1ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d20f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501ab6a0 con 0x56471ce4c1b0 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501a8f70 con 0x56471ce232f0 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.734+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501a6dc0 con 0x56471ce2b4d0 2024-05-10T21:09:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d20f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49346/0 (socket says 172.21.15.50:49346) 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 -- 172.21.15.50:0/2602466576 learned_addr learned my addr 172.21.15.50:0/2602466576 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d20f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52984/0 (socket says 172.21.15.50:52984) 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d4f400640 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d1ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 -- 172.21.15.50:0/2602466576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 msgr2=0x56471d1ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d1ed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 -- 172.21.15.50:0/2602466576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 msgr2=0x56471d20f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d20f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.735+0000 7f5d46a00640 1 -- 172.21.15.50:0/2602466576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471d1f2a80 con 0x56471ce2b4d0 2024-05-10T21:09:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.736+0000 7f5d46a00640 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009be0 tx=0x7f5d38009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43ee60e29edb53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.736+0000 7f5d3f400640 1 -- 172.21.15.50:0/2602466576 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38032070 con 0x56471ce2b4d0 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.736+0000 7f5d3f400640 1 -- 172.21.15.50:0/2602466576 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d380044f0 con 0x56471ce2b4d0 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.736+0000 7f5d3f400640 1 -- 172.21.15.50:0/2602466576 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005460 con 0x56471ce2b4d0 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/2602466576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 msgr2=0x56471d1ed6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009be0 tx=0x7f5d38009310 comp rx=0 tx=0).stop 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/2602466576 shutdown_connections 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce2b4d0 0x56471d1ed6e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce4c1b0 0x56471d20f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/2602466576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d1ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/2602466576 >> 172.21.15.50:0/2602466576 conn(0x56471ce40300 msgr2=0x56471d1eca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/2602466576 shutdown_connections 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.737+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/2602466576 wait complete. 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.738+0000 7f5d5cfa8740 1 -- start start 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.738+0000 7f5d5cfa8740 1 Processor -- start 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d202970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 0x56471d202eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501a7430 con 0x56471ce2b4d0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501a88a0 con 0x56471ce232f0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501a7b00 con 0x56471ce4c1b0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d5cfa8740 10 client.0 _ll_register_callbacks cb 0x7f5d4000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 0x56471d202eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d202970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 0x56471d202eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52994/0 (socket says 172.21.15.50:52994) 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d46a00640 1 -- 172.21.15.50:0/1369801737 learned_addr learned my addr 172.21.15.50:0/1369801737 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4fe00640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 -- 172.21.15.50:0/1369801737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 msgr2=0x56471d2033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 -- 172.21.15.50:0/1369801737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 msgr2=0x56471d202eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 0x56471d202eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.739+0000 7f5d4f400640 1 -- 172.21.15.50:0/1369801737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471d270330 con 0x56471ce232f0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d4f400640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d202970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d400a5bd0 tx=0x7f5d400e2940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df36e3671c3f27df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d400a9420 con 0x56471ce232f0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d40005000 con 0x56471ce232f0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d380044f0 con 0x56471ce232f0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d400a9600 con 0x56471ce232f0 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d5cfa8740 10 client.25487 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.740+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56471d2715c0 con 0x56471ce232f0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d380044f0 con 0x56471ce232f0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d400c4040 con 0x56471ce232f0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 10 client.25487 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 20 client.25487 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471cde01c0 0x56471d273c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 4 client.25487 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56471d159fd0 con 0x56471cde01c0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.741+0000 7f5d46a00640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471cde01c0 0x56471d273c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.742+0000 7f5d46a00640 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471cde01c0 0x56471d273c90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8179e2de4c557e86 server_cookie=543313307996ba7c in_seq=0 out_seq=0 2024-05-10T21:09:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.742+0000 7f5d3cc00640 10 client.25487 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d3cc00640 1 -- 172.21.15.50:0/1369801737 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56471d159fd0 con 0x56471cde01c0 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d3cc00640 10 client.25487 handle_command_reply: tid=0 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 client.25487 shutdown 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 msgr2=0x56471d202970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d202970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d400a5bd0 tx=0x7f5d400e2940 comp rx=0 tx=0).stop 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 shutdown_connections 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56471cde01c0 0x56471d273c90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56471ce4c1b0 0x56471d2033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56471ce2b4d0 0x56471d202eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 --2- 172.21.15.50:0/1369801737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56471ce232f0 0x56471d202970 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d3cc00640 0 client.25487 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:09:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 >> 172.21.15.50:0/1369801737 conn(0x56471ce40300 msgr2=0x56471d1ed4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 shutdown_connections 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.746+0000 7f5d5cfa8740 1 -- 172.21.15.50:0/1369801737 wait complete. 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d5cfa8740 20 client.25487 trim_cache size 0 max 16384 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:09:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 msgr2=0x7f5d340b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 0x7f5d340b4cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x56471d1f38e0 tx=0x7f5d48031af0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 msgr2=0x7f5d5019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000e9f0 tx=0x7f5d4000eeb0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 shutdown_connections 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b27e0 0x7f5d340b4cf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d50109820 0x7f5d501a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d50106ec0 0x7f5d5019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 --2- 172.21.15.50:0/3171285357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d50104560 0x7f5d5019a170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.747+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 >> 172.21.15.50:0/3171285357 conn(0x7f5d500fdb60 msgr2=0x7f5d50105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.748+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 shutdown_connections 2024-05-10T21:09:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:09:53.748+0000 7f5d54a00640 1 -- 172.21.15.50:0/3171285357 wait complete. 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 -- 172.21.15.50:0/911789193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c10c5c0 msgr2=0x7fec7c10c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 --2- 172.21.15.50:0/911789193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c10c5c0 0x7fec7c10c9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009990 tx=0x7fec6c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 -- 172.21.15.50:0/911789193 shutdown_connections 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 --2- 172.21.15.50:0/911789193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c106ed0 0x7fec7c113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 --2- 172.21.15.50:0/911789193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c10c5c0 0x7fec7c10c9d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 --2- 172.21.15.50:0/911789193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106580 0x7fec7c106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 -- 172.21.15.50:0/911789193 >> 172.21.15.50:0/911789193 conn(0x7fec7c102080 msgr2=0x7fec7c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.844+0000 7fec81800700 1 -- 172.21.15.50:0/911789193 shutdown_connections 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.845+0000 7fec81800700 1 -- 172.21.15.50:0/911789193 wait complete. 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 Processor -- start 2024-05-10T21:09:54.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 -- start start 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106ed0 0x7fec7c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c10c5c0 0x7fec7c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c116720 con 0x7fec7c106580 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1165a0 con 0x7fec7c106ed0 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.846+0000 7fec81800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1168a0 con 0x7fec7c10c5c0 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44732/0 (socket says 172.21.15.50:44732) 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 -- 172.21.15.50:0/3567994865 learned_addr learned my addr 172.21.15.50:0/3567994865 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7b400700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c10c5c0 0x7fec7c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7a000700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106ed0 0x7fec7c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 -- 172.21.15.50:0/3567994865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c10c5c0 msgr2=0x7fec7c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c10c5c0 0x7fec7c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 -- 172.21.15.50:0/3567994865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106ed0 msgr2=0x7fec7c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106ed0 0x7fec7c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.847+0000 7fec7aa00700 1 -- 172.21.15.50:0/3567994865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1a2000 con 0x7fec7c106580 2024-05-10T21:09:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec7aa00700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006010 tx=0x7fec6c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec73400700 1 -- 172.21.15.50:0/3567994865 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00fb00 con 0x7fec7c106580 2024-05-10T21:09:54.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec73400700 1 -- 172.21.15.50:0/3567994865 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c00fca0 con 0x7fec7c106580 2024-05-10T21:09:54.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec73400700 1 -- 172.21.15.50:0/3567994865 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c017650 con 0x7fec7c106580 2024-05-10T21:09:54.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c1a2290 con 0x7fec7c106580 2024-05-10T21:09:54.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.848+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1a26c0 con 0x7fec7c106580 2024-05-10T21:09:54.849 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:54 smithi050 ceph-mon[29546]: pgmap v491: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-10T21:09:54.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.852+0000 7fec73400700 1 -- 172.21.15.50:0/3567994865 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec6c009d10 con 0x7fec7c106580 2024-05-10T21:09:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.853+0000 7fec9025db80 1 Processor -- start 2024-05-10T21:09:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.853+0000 7fec9025db80 1 -- start start 2024-05-10T21:09:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be55654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be5565e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561be5487a20 con 0x561be53e7e40 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561be5425760 con 0x561be556b830 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec9025db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561be54839e0 con 0x561be55659f0 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec73400700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 0x7fec600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec73400700 1 -- 172.21.15.50:0/3567994865 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fec6c0e3350 con 0x7fec7c106580 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be55654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec7a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be5565e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be55654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39036/0 (socket says 172.21.15.50:39036) 2024-05-10T21:09:54.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec7aa00700 1 -- 172.21.15.50:0/114243438 learned_addr learned my addr 172.21.15.50:0/114243438 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:54.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.854+0000 7fec7b400700 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7b400700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 0x7fec600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 -- 172.21.15.50:0/114243438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 msgr2=0x561be55654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be55654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 -- 172.21.15.50:0/114243438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 msgr2=0x561be556b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 -- 172.21.15.50:0/114243438 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be53d48d0 con 0x561be55659f0 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec7a000700 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be5565e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec7c06d120 tx=0x7fec68009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06988b17f78208c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.855+0000 7fec67e00700 1 -- 172.21.15.50:0/114243438 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68011c80 con 0x561be55659f0 2024-05-10T21:09:54.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.856+0000 7fec7b400700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 0x7fec600b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec74009ec0 tx=0x7fec7400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.856+0000 7fec67e00700 1 -- 172.21.15.50:0/114243438 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68011e20 con 0x561be55659f0 2024-05-10T21:09:54.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.856+0000 7fec67e00700 1 -- 172.21.15.50:0/114243438 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68005280 con 0x561be55659f0 2024-05-10T21:09:54.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.856+0000 7fec9025db80 1 -- 172.21.15.50:0/114243438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 msgr2=0x561be5565e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.856+0000 7fec9025db80 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be5565e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec7c06d120 tx=0x7fec68009f60 comp rx=0 tx=0).stop 2024-05-10T21:09:54.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 -- 172.21.15.50:0/114243438 shutdown_connections 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be5565e00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 --2- 172.21.15.50:0/114243438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be55654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 -- 172.21.15.50:0/114243438 >> 172.21.15.50:0/114243438 conn(0x561be542cf70 msgr2=0x561be55602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 -- 172.21.15.50:0/114243438 shutdown_connections 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.857+0000 7fec9025db80 1 -- 172.21.15.50:0/114243438 wait complete. 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.858+0000 7fec9025db80 1 -- start start 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.858+0000 7fec9025db80 1 Processor -- start 2024-05-10T21:09:54.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.858+0000 7fec9025db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.858+0000 7fec9025db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be556f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec9025db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be556fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec9025db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561be5572310 con 0x561be53e7e40 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec9025db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561be53edb20 con 0x561be55659f0 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec9025db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561be5572490 con 0x561be556b830 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec9025db80 10 client.0 _ll_register_callbacks cb 0x561be542ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be556f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec7b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be556fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44752/0 (socket says 172.21.15.50:44752) 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.859+0000 7fec7aa00700 1 -- 172.21.15.50:0/2148840200 learned_addr learned my addr 172.21.15.50:0/2148840200 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 -- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 msgr2=0x561be556fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be556fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 -- 172.21.15.50:0/2148840200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 msgr2=0x561be556f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be556f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 -- 172.21.15.50:0/2148840200 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be544d420 con 0x561be53e7e40 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7b400700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be556fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0a6a00 tx=0x7fec6c0aaf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d907b21fac65dce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec64c00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0aa140 con 0x561be53e7e40 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fec6c0b1280 con 0x561be53e7e40 2024-05-10T21:09:54.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.860+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec68011e20 con 0x561be53e7e40 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec6c0a6160 con 0x561be53e7e40 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 10 client.15381 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561be557ce70 con 0x561be53e7e40 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68011e20 con 0x561be53e7e40 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fec6c0a9c30 con 0x561be53e7e40 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 10 client.15381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 20 client.15381 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be557f3e0 0x561be5581870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.861+0000 7fec9025db80 4 client.15381 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.862+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be55321c0 con 0x561be557f3e0 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.862+0000 7fec7a000700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be557f3e0 0x561be5581870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.863+0000 7fec64c00700 10 client.15381 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.863+0000 7fec7a000700 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be557f3e0 0x561be5581870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dcc9abb50541383 server_cookie=1fad248ac1c01353 in_seq=0 out_seq=0 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec64c00700 1 -- 172.21.15.50:0/2148840200 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561be55321c0 con 0x561be557f3e0 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec64c00700 10 client.15381 handle_command_reply: tid=0 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 client.15381 shutdown 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 msgr2=0x561be556f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0a6a00 tx=0x7fec6c0aaf10 comp rx=0 tx=0).stop 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 shutdown_connections 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be557f3e0 0x561be5581870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be556fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be556f010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 --2- 172.21.15.50:0/2148840200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be556f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 >> 172.21.15.50:0/2148840200 conn(0x561be542cf70 msgr2=0x561be55644a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.864+0000 7fec64c00700 0 client.15381 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.865+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 shutdown_connections 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.865+0000 7fec9025db80 1 -- 172.21.15.50:0/2148840200 wait complete. 2024-05-10T21:09:54.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.865+0000 7fec9025db80 20 client.15381 trim_cache size 0 max 16384 2024-05-10T21:09:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 Processor -- start 2024-05-10T21:09:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 -- start start 2024-05-10T21:09:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be556e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be556a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1b3630 con 0x561be53e7e40 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1b0ec0 con 0x561be556b830 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.888+0000 7fec9025db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1aed30 con 0x561be55659f0 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be556e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be556a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be556a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39052/0 (socket says 172.21.15.50:39052) 2024-05-10T21:09:54.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7b400700 1 -- 172.21.15.50:0/2433115535 learned_addr learned my addr 172.21.15.50:0/2433115535 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be556e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54448/0 (socket says 172.21.15.50:54448) 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7aa00700 1 -- 172.21.15.50:0/2433115535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 msgr2=0x561be556e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be556e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7aa00700 1 -- 172.21.15.50:0/2433115535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 msgr2=0x561be556a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.889+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be556a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.890+0000 7fec7aa00700 1 -- 172.21.15.50:0/2433115535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be55583e0 con 0x561be53e7e40 2024-05-10T21:09:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.890+0000 7fec7aa00700 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55654b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0aeda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5103c42ed87c1c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.890+0000 7fec67400700 1 -- 172.21.15.50:0/2433115535 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0aa140 con 0x561be53e7e40 2024-05-10T21:09:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.890+0000 7fec67400700 1 -- 172.21.15.50:0/2433115535 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c00fe50 con 0x561be53e7e40 2024-05-10T21:09:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.890+0000 7fec67400700 1 -- 172.21.15.50:0/2433115535 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0aa7a0 con 0x561be53e7e40 2024-05-10T21:09:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 -- 172.21.15.50:0/2433115535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 msgr2=0x561be55654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55654b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0aeda0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 -- 172.21.15.50:0/2433115535 shutdown_connections 2024-05-10T21:09:54.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be556b830 0x561be556a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55654b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 --2- 172.21.15.50:0/2433115535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be55659f0 0x561be556e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 -- 172.21.15.50:0/2433115535 >> 172.21.15.50:0/2433115535 conn(0x561be542cf70 msgr2=0x561be555a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 -- 172.21.15.50:0/2433115535 shutdown_connections 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.891+0000 7fec9025db80 1 -- 172.21.15.50:0/2433115535 wait complete. 2024-05-10T21:09:54.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.892+0000 7fec9025db80 1 -- start start 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.892+0000 7fec9025db80 1 Processor -- start 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be5577c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be5586060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1af380 con 0x561be53e7e40 2024-05-10T21:09:54.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1b07f0 con 0x561be55659f0 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1afa50 con 0x561be556b830 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec9025db80 10 client.0 _ll_register_callbacks cb 0x561be542ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec7b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be5586060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44778/0 (socket says 172.21.15.50:44778) 2024-05-10T21:09:54.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.893+0000 7fec7aa00700 1 -- 172.21.15.50:0/917696674 learned_addr learned my addr 172.21.15.50:0/917696674 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7a000700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be5577c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 -- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 msgr2=0x561be5586060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be5586060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 -- 172.21.15.50:0/917696674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 msgr2=0x561be5577c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:54.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be5577c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 -- 172.21.15.50:0/917696674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be5586740 con 0x561be53e7e40 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec7aa00700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0a9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f076e077d73e137b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec64c00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c06e070 con 0x561be53e7e40 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.894+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6c00fe50 con 0x561be53e7e40 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fec6c0b1280 con 0x561be53e7e40 2024-05-10T21:09:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec6c0aaa20 con 0x561be53e7e40 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec9025db80 10 client.15387 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561be5586740 con 0x561be53e7e40 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c00fe50 con 0x561be53e7e40 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fec6c022470 con 0x561be53e7e40 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec9025db80 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.895+0000 7fec9025db80 10 client.15387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:09:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.896+0000 7fec9025db80 20 client.15387 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.896+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be55e97a0 0x561be55ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.896+0000 7fec9025db80 4 client.15387 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:09:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.896+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561be5495400 con 0x561be55e97a0 2024-05-10T21:09:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.896+0000 7fec7a000700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be55e97a0 0x561be55ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec7a000700 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be55e97a0 0x561be55ebbc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=678d771d52d9f7d8 server_cookie=75f983f548a886b9 in_seq=0 out_seq=0 2024-05-10T21:09:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec64c00700 10 client.15387 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec64c00700 1 -- 172.21.15.50:0/917696674 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561be5495400 con 0x561be55e97a0 2024-05-10T21:09:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec64c00700 10 client.15387 handle_command_reply: tid=0 2024-05-10T21:09:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 client.15387 shutdown 2024-05-10T21:09:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 msgr2=0x561be55776e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0a9ee0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 shutdown_connections 2024-05-10T21:09:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561be55e97a0 0x561be55ebbc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be556b830 0x561be5586060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561be53e7e40 0x561be55776e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 --2- 172.21.15.50:0/917696674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561be55659f0 0x561be5577c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec64c00700 0 client.15387 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:09:54.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.897+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 >> 172.21.15.50:0/917696674 conn(0x561be542cf70 msgr2=0x561be556e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.910 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:09:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.898+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 shutdown_connections 2024-05-10T21:09:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.898+0000 7fec9025db80 1 -- 172.21.15.50:0/917696674 wait complete. 2024-05-10T21:09:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.898+0000 7fec9025db80 20 client.15387 trim_cache size 0 max 16384 2024-05-10T21:09:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 msgr2=0x7fec600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 0x7fec600b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec74009ec0 tx=0x7fec7400b040 comp rx=0 tx=0).stop 2024-05-10T21:09:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 msgr2=0x7fec7c19d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006010 tx=0x7fec6c0057c0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 shutdown_connections 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fec600b2880 0x7fec600b4d90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec7c10c5c0 0x7fec7c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec7c106580 0x7fec7c19d130 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 --2- 172.21.15.50:0/3567994865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec7c106ed0 0x7fec7c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 >> 172.21.15.50:0/3567994865 conn(0x7fec7c102080 msgr2=0x7fec7c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 shutdown_connections 2024-05-10T21:09:54.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:54.899+0000 7fec81800700 1 -- 172.21.15.50:0/3567994865 wait complete. 2024-05-10T21:09:55.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:54 smithi092 ceph-mon[34415]: pgmap v491: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-10T21:09:55.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:54 smithi142 ceph-mon[34846]: pgmap v491: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-10T21:09:55.872 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:09:56.380 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:09:57.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:56 smithi092 ceph-mon[34415]: pgmap v492: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:09:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:56 smithi142 ceph-mon[34846]: pgmap v492: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:09:57.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:56 smithi050 ceph-mon[29546]: pgmap v492: 97 pgs: 97 active+clean; 671 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:09:58.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 -- 172.21.15.50:0/3289759342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 msgr2=0x7ffb94109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 --2- 172.21.15.50:0/3289759342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94109140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb800099c0 tx=0x7ffb80009cd0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 -- 172.21.15.50:0/3289759342 shutdown_connections 2024-05-10T21:09:58.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 --2- 172.21.15.50:0/3289759342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94104400 0x7ffb94104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 --2- 172.21.15.50:0/3289759342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94109680 0x7ffb94110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 --2- 172.21.15.50:0/3289759342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94109140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.824+0000 7ffb99e00700 1 -- 172.21.15.50:0/3289759342 >> 172.21.15.50:0/3289759342 conn(0x7ffb94079d10 msgr2=0x7ffb9407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.825+0000 7ffb99e00700 1 -- 172.21.15.50:0/3289759342 shutdown_connections 2024-05-10T21:09:58.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.825+0000 7ffb99e00700 1 -- 172.21.15.50:0/3289759342 wait complete. 2024-05-10T21:09:58.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:09:58 smithi050 ceph-mon[29546]: pgmap v493: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.826+0000 7ffb99e00700 1 Processor -- start 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.826+0000 7ffb99e00700 1 -- start start 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94104400 0x7ffb94075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94109680 0x7ffb94076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94114300 con 0x7ffb94104400 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94114180 con 0x7ffb94109680 2024-05-10T21:09:58.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94114480 con 0x7ffb94108d30 2024-05-10T21:09:58.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb98a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94104400 0x7ffb94075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54466/0 (socket says 172.21.15.50:54466) 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 -- 172.21.15.50:0/2672268451 learned_addr learned my addr 172.21.15.50:0/2672268451 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb99400700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94109680 0x7ffb94076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 -- 172.21.15.50:0/2672268451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94109680 msgr2=0x7ffb94076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94109680 0x7ffb94076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 -- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94104400 msgr2=0x7ffb94075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.827+0000 7ffb93e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94104400 0x7ffb94075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.828+0000 7ffb93e00700 1 -- 172.21.15.50:0/2672268451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941a9050 con 0x7ffb94108d30 2024-05-10T21:09:58.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.828+0000 7ffb93e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005f50 tx=0x7ffb80005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.829+0000 7ffb91600700 1 -- 172.21.15.50:0/2672268451 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80026070 con 0x7ffb94108d30 2024-05-10T21:09:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.829+0000 7ffb91600700 1 -- 172.21.15.50:0/2672268451 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb80003c70 con 0x7ffb94108d30 2024-05-10T21:09:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.829+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb941a92e0 con 0x7ffb94108d30 2024-05-10T21:09:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.829+0000 7ffb91600700 1 -- 172.21.15.50:0/2672268451 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000f960 con 0x7ffb94108d30 2024-05-10T21:09:58.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.829+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb941a9790 con 0x7ffb94108d30 2024-05-10T21:09:58.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.833+0000 7ffb91600700 1 -- 172.21.15.50:0/2672268451 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffb80029070 con 0x7ffb94108d30 2024-05-10T21:09:58.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 Processor -- start 2024-05-10T21:09:58.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 -- start start 2024-05-10T21:09:58.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338db8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b338ccea20 con 0x55b338dad690 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b338c6c760 con 0x55b338dad280 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.834+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b338cca9e0 con 0x55b338c2ee40 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338db8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54472/0 (socket says 172.21.15.50:54472) 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 -- 172.21.15.50:0/848445784 learned_addr learned my addr 172.21.15.50:0/848445784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb98a00700 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb91600700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 -- 172.21.15.50:0/848445784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 msgr2=0x55b338db68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 -- 172.21.15.50:0/848445784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 msgr2=0x55b338db8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338db8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb99400700 1 -- 172.21.15.50:0/848445784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b338c1b8d0 con 0x55b338c2ee40 2024-05-10T21:09:58.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.835+0000 7ffb98a00700 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:58.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.836+0000 7ffb91600700 1 -- 172.21.15.50:0/2672268451 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffb800e2a00 con 0x7ffb94108d30 2024-05-10T21:09:58.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.836+0000 7ffb99400700 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c011680 tx=0x7ffb8c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebb70916ccbbefb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.836+0000 7ffb86000700 1 -- 172.21.15.50:0/848445784 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c009190 con 0x55b338c2ee40 2024-05-10T21:09:58.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.836+0000 7ffb99400700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.837+0000 7ffb86000700 1 -- 172.21.15.50:0/848445784 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8c01a020 con 0x55b338c2ee40 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.837+0000 7ffb86000700 1 -- 172.21.15.50:0/848445784 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c019040 con 0x55b338c2ee40 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.837+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/848445784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 msgr2=0x55b338dacd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.837+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c011680 tx=0x7ffb8c009920 comp rx=0 tx=0).stop 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.837+0000 7ffb99400700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c007940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/848445784 shutdown_connections 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338db8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dacd40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/848445784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/848445784 >> 172.21.15.50:0/848445784 conn(0x55b338c6da80 msgr2=0x55b338da7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/848445784 shutdown_connections 2024-05-10T21:09:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.838+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/848445784 wait complete. 2024-05-10T21:09:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 -- start start 2024-05-10T21:09:58.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 Processor -- start 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 0x55b338db62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338dad690 0x55b338db6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b338db97a0 con 0x55b338c2ee40 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b338c34b20 con 0x55b338dad280 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.839+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b338db9920 con 0x55b338dad690 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffba9e9ab80 10 client.0 _ll_register_callbacks cb 0x55b338c62380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb98a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 0x55b338db62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39082/0 (socket says 172.21.15.50:39082) 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 -- 172.21.15.50:0/3640200208 learned_addr learned my addr 172.21.15.50:0/3640200208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb98a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 0x55b338db62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44810/0 (socket says 172.21.15.50:44810) 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb99400700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338dad690 0x55b338db6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 -- 172.21.15.50:0/3640200208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338dad690 msgr2=0x55b338db6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338dad690 0x55b338db6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 -- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 msgr2=0x55b338db62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 0x55b338db62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.840+0000 7ffb93e00700 1 -- 172.21.15.50:0/3640200208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b338c94420 con 0x55b338dad280 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb93e00700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c00 tx=0x7ffb800aa5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a046fe98ab8a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800a99d0 con 0x55b338dad280 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffb800e05e0 con 0x55b338dad280 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8c01a020 con 0x55b338dad280 2024-05-10T21:09:58.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb800a97f0 con 0x55b338dad280 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffba9e9ab80 10 client.25517 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b338dc43a0 con 0x55b338dad280 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.841+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8c01a020 con 0x55b338dad280 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ffb800ae0b0 con 0x55b338dad280 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 10 client.25517 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 10 client.25517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 20 client.25517 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338dc6870 0x55b338dc8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 4 client.25517 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b338d79070 con 0x55b338dc6870 2024-05-10T21:09:58.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.842+0000 7ffb98a00700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338dc6870 0x55b338dc8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.843+0000 7ffb98a00700 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338dc6870 0x55b338dc8cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6e83eb7b73d11ff server_cookie=fa5bb8db4e1f94e4 in_seq=0 out_seq=0 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.843+0000 7ffb7aa00700 10 client.25517 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.844+0000 7ffb7aa00700 1 -- 172.21.15.50:0/3640200208 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b338d79070 con 0x55b338dc6870 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.844+0000 7ffb7aa00700 10 client.25517 handle_command_reply: tid=0 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.844+0000 7ffba9e9ab80 1 client.25517 shutdown 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.844+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 msgr2=0x55b338db6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.844+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c00 tx=0x7ffb800aa5c0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 shutdown_connections 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338dc6870 0x55b338dc8cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338dad690 0x55b338db6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338c2ee40 0x55b338db62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/3640200208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db6820 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 >> 172.21.15.50:0/3640200208 conn(0x55b338c6da80 msgr2=0x55b338dab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffb7aa00700 0 client.25517 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 shutdown_connections 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/3640200208 wait complete. 2024-05-10T21:09:58.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.845+0000 7ffba9e9ab80 20 client.25517 trim_cache size 0 max 16384 2024-05-10T21:09:58.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 Processor -- start 2024-05-10T21:09:58.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 -- start start 2024-05-10T21:09:58.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338dc2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941b3300 con 0x55b338dad690 2024-05-10T21:09:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941b0b90 con 0x55b338dad280 2024-05-10T21:09:58.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.863+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941aea00 con 0x55b338c2ee40 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54480/0 (socket says 172.21.15.50:54480) 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 -- 172.21.15.50:0/2094707943 learned_addr learned my addr 172.21.15.50:0/2094707943 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39092/0 (socket says 172.21.15.50:39092) 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb99400700 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338dc2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 -- 172.21.15.50:0/2094707943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 msgr2=0x55b338db17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 -- 172.21.15.50:0/2094707943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 msgr2=0x55b338dc2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338dc2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.864+0000 7ffb98a00700 1 -- 172.21.15.50:0/2094707943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b338dadaa0 con 0x55b338c2ee40 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.865+0000 7ffb98a00700 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55b338da8e70 tx=0x7ffb88009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fa2e73ca1839c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.865+0000 7ffb85600700 1 -- 172.21.15.50:0/2094707943 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88022020 con 0x55b338c2ee40 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.865+0000 7ffb85600700 1 -- 172.21.15.50:0/2094707943 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8801d8c0 con 0x55b338c2ee40 2024-05-10T21:09:58.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.865+0000 7ffb85600700 1 -- 172.21.15.50:0/2094707943 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8801e6d0 con 0x55b338c2ee40 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/2094707943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 msgr2=0x55b338dac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55b338da8e70 tx=0x7ffb88009260 comp rx=0 tx=0).stop 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/2094707943 shutdown_connections 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad690 0x55b338dc2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dac130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/2094707943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad280 0x55b338db17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/2094707943 >> 172.21.15.50:0/2094707943 conn(0x55b338c6da80 msgr2=0x55b338db6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/2094707943 shutdown_connections 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.866+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/2094707943 wait complete. 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.867+0000 7ffba9e9ab80 1 -- start start 2024-05-10T21:09:58.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.867+0000 7ffba9e9ab80 1 Processor -- start 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.867+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 0x55b338dbb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941af050 con 0x55b338dad280 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941b04c0 con 0x55b338dad690 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb941af720 con 0x55b338c2ee40 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffba9e9ab80 10 client.0 _ll_register_callbacks cb 0x55b338c62380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44826/0 (socket says 172.21.15.50:44826) 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 -- 172.21.15.50:0/4001517428 learned_addr learned my addr 172.21.15.50:0/4001517428 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:09:58.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb98a00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb99400700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 0x55b338dbb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 -- 172.21.15.50:0/4001517428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 msgr2=0x55b338dba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 -- 172.21.15.50:0/4001517428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 msgr2=0x55b338dbb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 0x55b338dbb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.868+0000 7ffb93e00700 1 -- 172.21.15.50:0/4001517428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b338e38870 con 0x55b338dad280 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb98a00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb99400700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 0x55b338dbb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb93e00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb800ae570 tx=0x7ffb80009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c766b1db30c640e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:09:58.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:09:58.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800094b0 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffb800037c0 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8801d8c0 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb800a9b90 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffba9e9ab80 10 client.15405 fetch_fsmap learned FSMap version 25 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.869+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b338e39930 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8801d8c0 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ffb800a59f0 con 0x55b338dad280 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 10 client.15405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 20 client.15405 populate_metadata read hostname 'smithi050' 2024-05-10T21:09:58.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338e39eb0 0x55b338e3c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 4 client.15405 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b338cdc400 con 0x55b338e39eb0 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.870+0000 7ffb98a00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338e39eb0 0x55b338e3c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffb98a00700 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338e39eb0 0x55b338e3c2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5455061462b19c87 server_cookie=202fb183b5a03ed3 in_seq=0 out_seq=0 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffb7aa00700 10 client.15405 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffb7aa00700 1 -- 172.21.15.50:0/4001517428 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b338cdc400 con 0x55b338e39eb0 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffb7aa00700 10 client.15405 handle_command_reply: tid=0 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffba9e9ab80 1 client.15405 shutdown 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 msgr2=0x55b338dbac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.871+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb800ae570 tx=0x7ffb80009380 comp rx=0 tx=0).stop 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 shutdown_connections 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b338dad690 0x55b338dbb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55b338e39eb0 0x55b338e3c2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b338c2ee40 0x55b338dba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 --2- 172.21.15.50:0/4001517428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b338dad280 0x55b338dbac30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffb7aa00700 0 client.15405 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 >> 172.21.15.50:0/4001517428 conn(0x55b338c6da80 msgr2=0x55b338db78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 shutdown_connections 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 1 -- 172.21.15.50:0/4001517428 wait complete. 2024-05-10T21:09:58.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.872+0000 7ffba9e9ab80 20 client.15405 trim_cache size 0 max 16384 2024-05-10T21:09:58.874 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:09:58.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 msgr2=0x7ffb7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c007940 comp rx=0 tx=0).stop 2024-05-10T21:09:58.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 msgr2=0x7ffb94076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005f50 tx=0x7ffb80005280 comp rx=0 tx=0).stop 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 shutdown_connections 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb94109680 0x7ffb94076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb94104400 0x7ffb94075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 --2- 172.21.15.50:0/2672268451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb94108d30 0x7ffb94076000 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 >> 172.21.15.50:0/2672268451 conn(0x7ffb94079d10 msgr2=0x7ffb94101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 shutdown_connections 2024-05-10T21:09:58.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:09:58.873+0000 7ffb99e00700 1 -- 172.21.15.50:0/2672268451 wait complete. 2024-05-10T21:09:59.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:09:58 smithi092 ceph-mon[34415]: pgmap v493: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:09:58 smithi142 ceph-mon[34846]: pgmap v493: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:09:59.825 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:10:00.300 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:01.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:00 smithi092 ceph-mon[34415]: pgmap v494: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:01.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:00 smithi092 ceph-mon[34415]: overall HEALTH_OK 2024-05-10T21:10:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:00 smithi142 ceph-mon[34846]: pgmap v494: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:00 smithi142 ceph-mon[34846]: overall HEALTH_OK 2024-05-10T21:10:01.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:00 smithi050 ceph-mon[29546]: pgmap v494: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:01.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:00 smithi050 ceph-mon[29546]: overall HEALTH_OK 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 -- 172.21.15.50:0/2189318560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 msgr2=0x7fc75c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 --2- 172.21.15.50:0/2189318560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c075f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc750009990 tx=0x7fc750009ca0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 -- 172.21.15.50:0/2189318560 shutdown_connections 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 --2- 172.21.15.50:0/2189318560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c115370 0x7fc75c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 --2- 172.21.15.50:0/2189318560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c075f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 --2- 172.21.15.50:0/2189318560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c0764b0 0x7fc75c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.747+0000 7fc763800700 1 -- 172.21.15.50:0/2189318560 >> 172.21.15.50:0/2189318560 conn(0x7fc75c100040 msgr2=0x7fc75c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.748+0000 7fc763800700 1 -- 172.21.15.50:0/2189318560 shutdown_connections 2024-05-10T21:10:02.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.748+0000 7fc763800700 1 -- 172.21.15.50:0/2189318560 wait complete. 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.749+0000 7fc763800700 1 Processor -- start 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.749+0000 7fc763800700 1 -- start start 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.749+0000 7fc763800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.749+0000 7fc763800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c0764b0 0x7fc75c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc763800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 0x7fc75c1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc763800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c11aad0 con 0x7fc75c075b60 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc763800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c11a950 con 0x7fc75c115370 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc763800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c11ac50 con 0x7fc75c0764b0 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc760e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 0x7fc75c1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc760e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44856/0 (socket says 172.21.15.50:44856) 2024-05-10T21:10:02.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc75be00700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c0764b0 0x7fc75c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc760e00700 1 -- 172.21.15.50:0/773766200 learned_addr learned my addr 172.21.15.50:0/773766200 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 -- 172.21.15.50:0/773766200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c0764b0 msgr2=0x7fc75c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c0764b0 0x7fc75c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 -- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 msgr2=0x7fc75c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.750+0000 7fc761800700 1 -- 172.21.15.50:0/773766200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c1a62b0 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc761800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 0x7fc75c1a5c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc75400efc0 tx=0x7fc75400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc759600700 1 -- 172.21.15.50:0/773766200 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540092e0 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1a6540 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc759600700 1 -- 172.21.15.50:0/773766200 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc754004bb0 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc759600700 1 -- 172.21.15.50:0/773766200 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754004020 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.751+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1adec0 con 0x7fc75c115370 2024-05-10T21:10:02.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.755+0000 7fc759600700 1 -- 172.21.15.50:0/773766200 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc75401b030 con 0x7fc75c115370 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc772316b80 1 Processor -- start 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc772316b80 1 -- start start 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc759600700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 0x7fc7440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc772316b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc772316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b1abf0 0x557b50b14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.757+0000 7fc772316b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 0x557b50b15060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc772316b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557b50a36a20 con 0x557b50996e40 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc772316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557b509d4760 con 0x557b50b1abf0 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc772316b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557b50a329e0 con 0x557b50b14c50 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc759600700 1 -- 172.21.15.50:0/773766200 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc7540e1c00 con 0x7fc75c115370 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc75be00700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 0x7fc7440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44862/0 (socket says 172.21.15.50:44862) 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 -- 172.21.15.50:0/439329916 learned_addr learned my addr 172.21.15.50:0/439329916 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc75be00700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b1abf0 0x557b50b14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc761800700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 0x557b50b15060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 -- 172.21.15.50:0/439329916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 msgr2=0x557b50b15060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 0x557b50b15060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 -- 172.21.15.50:0/439329916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b1abf0 msgr2=0x557b50b14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b1abf0 0x557b50b14710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.758+0000 7fc760e00700 1 -- 172.21.15.50:0/439329916 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b509838d0 con 0x557b50996e40 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc761800700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 0x557b50b15060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc75be00700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 0x7fc7440b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc75c103540 tx=0x7fc748009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc760e00700 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7500096d0 tx=0x7fc750005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd97d4841fac01d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc74e000700 1 -- 172.21.15.50:0/439329916 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750027070 con 0x557b50996e40 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc74e000700 1 -- 172.21.15.50:0/439329916 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75000bbd0 con 0x557b50996e40 2024-05-10T21:10:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.759+0000 7fc74e000700 1 -- 172.21.15.50:0/439329916 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750003d00 con 0x557b50996e40 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 -- 172.21.15.50:0/439329916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 msgr2=0x557b50b1a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7500096d0 tx=0x7fc750005f00 comp rx=0 tx=0).stop 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 -- 172.21.15.50:0/439329916 shutdown_connections 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b14c50 0x557b50b15060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50996e40 0x557b50b1a6b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 --2- 172.21.15.50:0/439329916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b1abf0 0x557b50b14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.760+0000 7fc772316b80 1 -- 172.21.15.50:0/439329916 >> 172.21.15.50:0/439329916 conn(0x557b509d5a80 msgr2=0x557b50b0f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.761+0000 7fc772316b80 1 -- 172.21.15.50:0/439329916 shutdown_connections 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.761+0000 7fc772316b80 1 -- 172.21.15.50:0/439329916 wait complete. 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.761+0000 7fc772316b80 1 -- start start 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.761+0000 7fc772316b80 1 Processor -- start 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557b50b21660 con 0x557b50b1abf0 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557b5099cb20 con 0x557b50b14c50 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557b50b217e0 con 0x557b50996e40 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc772316b80 10 client.0 _ll_register_callbacks cb 0x557b509ca380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.762+0000 7fc75be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc761800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39130/0 (socket says 172.21.15.50:39130) 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 -- 172.21.15.50:0/371608439 learned_addr learned my addr 172.21.15.50:0/371608439 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc760e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 -- 172.21.15.50:0/371608439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 msgr2=0x557b50b1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 -- 172.21.15.50:0/371608439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 msgr2=0x557b50b1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 -- 172.21.15.50:0/371608439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b509fc420 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc75be00700 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc748011ea0 tx=0x7fc74801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66db04634e8f9f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.763+0000 7fc742a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800fa40 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc74800fbe0 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75000bbd0 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc74800fde0 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc772316b80 10 client.25535 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b50b2c220 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.764+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75000bbd0 con 0x557b50b14c50 2024-05-10T21:10:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc74801f760 con 0x557b50b14c50 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 10 client.25535 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 20 client.25535 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b2e900 0x557b50b30d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 4 client.25535 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b50ae1070 con 0x557b50b2e900 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.765+0000 7fc760e00700 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b2e900 0x557b50b30d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.766+0000 7fc760e00700 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b2e900 0x557b50b30d20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf1403107980caa server_cookie=58e9d4eadb8d2ebb in_seq=0 out_seq=0 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.766+0000 7fc742a00700 10 client.25535 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc742a00700 1 -- 172.21.15.50:0/371608439 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b50ae1070 con 0x557b50b2e900 2024-05-10T21:10:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc742a00700 10 client.25535 handle_command_reply: tid=0 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 client.25535 shutdown 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 msgr2=0x557b50b1e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc748011ea0 tx=0x7fc74801d000 comp rx=0 tx=0).stop 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 shutdown_connections 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b2e900 0x557b50b30d20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc742a00700 0 client.25535 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 --2- 172.21.15.50:0/371608439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b1e8c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 >> 172.21.15.50:0/371608439 conn(0x557b509d5a80 msgr2=0x557b50b13700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 shutdown_connections 2024-05-10T21:10:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.767+0000 7fc772316b80 1 -- 172.21.15.50:0/371608439 wait complete. 2024-05-10T21:10:02.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.768+0000 7fc772316b80 20 client.25535 trim_cache size 0 max 16384 2024-05-10T21:10:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.792+0000 7fc772316b80 1 Processor -- start 2024-05-10T21:10:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.792+0000 7fc772316b80 1 -- start start 2024-05-10T21:10:02.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b19bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b78d0 con 0x557b50b1abf0 2024-05-10T21:10:02.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b5160 con 0x557b50b14c50 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc772316b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b3010 con 0x557b50996e40 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc75be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc760e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc75be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39142/0 (socket says 172.21.15.50:39142) 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc75be00700 1 -- 172.21.15.50:0/533596389 learned_addr learned my addr 172.21.15.50:0/533596389 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.793+0000 7fc761800700 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b19bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 -- 172.21.15.50:0/533596389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 msgr2=0x557b50b14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b14710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 -- 172.21.15.50:0/533596389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 msgr2=0x557b50b19bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b19bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 -- 172.21.15.50:0/533596389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b50b1c910 con 0x557b50b14c50 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc75be00700 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc748005950 tx=0x7fc74800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c04950e31b5715 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.794+0000 7fc74d600700 1 -- 172.21.15.50:0/533596389 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480045f0 con 0x557b50b14c50 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.795+0000 7fc74d600700 1 -- 172.21.15.50:0/533596389 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc748037070 con 0x557b50b14c50 2024-05-10T21:10:02.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.795+0000 7fc74d600700 1 -- 172.21.15.50:0/533596389 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748003a00 con 0x557b50b14c50 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 -- 172.21.15.50:0/533596389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 msgr2=0x557b50b18b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc748005950 tx=0x7fc74800f6a0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 -- 172.21.15.50:0/533596389 shutdown_connections 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b1abf0 0x557b50b19bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50996e40 0x557b50b14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 --2- 172.21.15.50:0/533596389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50b14c50 0x557b50b18b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 -- 172.21.15.50:0/533596389 >> 172.21.15.50:0/533596389 conn(0x557b509d5a80 msgr2=0x557b50b0ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 -- 172.21.15.50:0/533596389 shutdown_connections 2024-05-10T21:10:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.796+0000 7fc772316b80 1 -- 172.21.15.50:0/533596389 wait complete. 2024-05-10T21:10:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.797+0000 7fc772316b80 1 -- start start 2024-05-10T21:10:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.797+0000 7fc772316b80 1 Processor -- start 2024-05-10T21:10:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50996e40 0x557b50b27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b14c50 0x557b50b35ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b3660 con 0x557b50b14c50 2024-05-10T21:10:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b4a90 con 0x557b50996e40 2024-05-10T21:10:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b3cf0 con 0x557b50b1abf0 2024-05-10T21:10:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.798+0000 7fc772316b80 10 client.0 _ll_register_callbacks cb 0x557b509ca380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc75be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b14c50 0x557b50b35ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54534/0 (socket says 172.21.15.50:54534) 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 -- 172.21.15.50:0/2999611642 learned_addr learned my addr 172.21.15.50:0/2999611642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc760e00700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50996e40 0x557b50b27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 -- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50996e40 msgr2=0x557b50b27140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50996e40 0x557b50b27140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 -- 172.21.15.50:0/2999611642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b14c50 msgr2=0x557b50b35ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b14c50 0x557b50b35ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.799+0000 7fc761800700 1 -- 172.21.15.50:0/2999611642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b50b367e0 con 0x557b50b1abf0 2024-05-10T21:10:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc761800700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc75400c6f0 tx=0x7fc7540b1ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e135758174bf6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc742a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540a8d90 con 0x557b50b1abf0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc7540a86f0 con 0x557b50b1abf0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc748037070 con 0x557b50b1abf0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7540a9980 con 0x557b50b1abf0 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc772316b80 10 client.25528 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.800+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b50b99070 con 0x557b50b1abf0 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc748037070 con 0x557b50b1abf0 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc7540a4bf0 con 0x557b50b1abf0 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 20 client.25528 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b99780 0x557b50b9bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 4 client.25528 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:02.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.801+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b50a44400 con 0x557b50b99780 2024-05-10T21:10:02.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.802+0000 7fc760e00700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b99780 0x557b50b9bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:02.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc760e00700 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b99780 0x557b50b9bba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e887dfbabec6dc10 server_cookie=11deaef61de6c917 in_seq=0 out_seq=0 2024-05-10T21:10:02.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc742a00700 10 client.25528 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc742a00700 1 -- 172.21.15.50:0/2999611642 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b50a44400 con 0x557b50b99780 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc742a00700 10 client.25528 handle_command_reply: tid=0 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 client.25528 shutdown 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 msgr2=0x557b50b36070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc75400c6f0 tx=0x7fc7540b1ea0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 shutdown_connections 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557b50b99780 0x557b50b9bba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557b50996e40 0x557b50b27140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557b50b1abf0 0x557b50b36070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 --2- 172.21.15.50:0/2999611642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b50b14c50 0x557b50b35ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 >> 172.21.15.50:0/2999611642 conn(0x557b509d5a80 msgr2=0x557b50b18860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.803+0000 7fc742a00700 0 client.25528 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.804+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 shutdown_connections 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.804+0000 7fc772316b80 1 -- 172.21.15.50:0/2999611642 wait complete. 2024-05-10T21:10:02.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.804+0000 7fc772316b80 20 client.25528 trim_cache size 0 max 16384 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.805+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 msgr2=0x7fc7440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.805+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 0x7fc7440b4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc75c103540 tx=0x7fc748009bf0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.805+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 msgr2=0x7fc75c1a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.805+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 0x7fc75c1a5c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc75400efc0 tx=0x7fc75400c3a0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 shutdown_connections 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc7440b2900 0x7fc7440b4e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc75c115370 0x7fc75c1a5c10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc75c075b60 0x7fc75c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 --2- 172.21.15.50:0/773766200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc75c0764b0 0x7fc75c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.806+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 >> 172.21.15.50:0/773766200 conn(0x7fc75c100040 msgr2=0x7fc75c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.807+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 shutdown_connections 2024-05-10T21:10:02.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:02.807+0000 7fc763800700 1 -- 172.21.15.50:0/773766200 wait complete. 2024-05-10T21:10:03.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:02 smithi092 ceph-mon[34415]: pgmap v495: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:03.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:02 smithi142 ceph-mon[34846]: pgmap v495: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:03.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:02 smithi050 ceph-mon[29546]: pgmap v495: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:04.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:10:04.735 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:10:05.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:04 smithi092 ceph-mon[34415]: pgmap v496: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:10:05.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:04 smithi142 ceph-mon[34846]: pgmap v496: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:10:05.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:04 smithi050 ceph-mon[29546]: pgmap v496: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T21:10:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:06.912 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:06 smithi050 ceph-mon[29546]: pgmap v497: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:10:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 -- 172.21.15.50:0/1594845655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 msgr2=0x7f89c81050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 --2- 172.21.15.50:0/1594845655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c81050d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009900 tx=0x7f89b402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 -- 172.21.15.50:0/1594845655 shutdown_connections 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 --2- 172.21.15.50:0/1594845655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c81099c0 0x7f89c810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 --2- 172.21.15.50:0/1594845655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c81050d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 --2- 172.21.15.50:0/1594845655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c8103a80 0x7f89c8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 -- 172.21.15.50:0/1594845655 >> 172.21.15.50:0/1594845655 conn(0x7f89c80fec70 msgr2=0x7f89c8101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 -- 172.21.15.50:0/1594845655 shutdown_connections 2024-05-10T21:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.036+0000 7f89cf000640 1 -- 172.21.15.50:0/1594845655 wait complete. 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.037+0000 7f89cf000640 1 Processor -- start 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 -- start start 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c8103a80 0x7f89c8076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c8075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810f5b0 con 0x7f89c81099c0 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810f430 con 0x7f89c8104cc0 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cf000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810f730 con 0x7f89c8103a80 2024-05-10T21:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89ce600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89ce600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35866/0 (socket says 172.21.15.50:35866) 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89ce600640 1 -- 172.21.15.50:0/3095749259 learned_addr learned my addr 172.21.15.50:0/3095749259 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cd200640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c8075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.038+0000 7f89cdc00640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c8103a80 0x7f89c8076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 -- 172.21.15.50:0/3095749259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c8103a80 msgr2=0x7f89c8076980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c8103a80 0x7f89c8076980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 -- 172.21.15.50:0/3095749259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 msgr2=0x7f89c8075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c8075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 -- 172.21.15.50:0/3095749259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c8075c90 con 0x7f89c81099c0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.039+0000 7f89ce600640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f89c000eb10 tx=0x7f89c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.040+0000 7f89c6000640 1 -- 172.21.15.50:0/3095749259 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0004b20 con 0x7f89c81099c0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.040+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c81a4210 con 0x7f89c81099c0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.040+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c81a4700 con 0x7f89c81099c0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.040+0000 7f89c6000640 1 -- 172.21.15.50:0/3095749259 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c0004cc0 con 0x7f89c81099c0 2024-05-10T21:10:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.040+0000 7f89c6000640 1 -- 172.21.15.50:0/3095749259 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0010690 con 0x7f89c81099c0 2024-05-10T21:10:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.045+0000 7f89d75c0740 1 Processor -- start 2024-05-10T21:10:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.047+0000 7f89c6000640 1 -- 172.21.15.50:0/3095749259 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f89c0005000 con 0x7f89c81099c0 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.047+0000 7f89d75c0740 1 -- start start 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 0x55569278a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923c74d0 0x555692791780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5556926ee550 con 0x5556923c74d0 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555692693bc0 con 0x5556923bf2f0 2024-05-10T21:10:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89d75c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5556926ea510 con 0x5556923e81b0 2024-05-10T21:10:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89cd200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89cd200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50972/0 (socket says 172.21.15.50:50972) 2024-05-10T21:10:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89cd200640 1 -- 172.21.15.50:0/1434121658 learned_addr learned my addr 172.21.15.50:0/1434121658 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.048+0000 7f89cd200640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923c74d0 0x555692791780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 0x55569278a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 -- 172.21.15.50:0/1434121658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 msgr2=0x55569278a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 -- 172.21.15.50:0/1434121658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923c74d0 msgr2=0x555692791780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923c74d0 0x555692791780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cdc00640 1 -- 172.21.15.50:0/1434121658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555692400210 con 0x5556923bf2f0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89c6000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 0x7f89b00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89cd200640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.049+0000 7f89c6000640 1 -- 172.21.15.50:0/3095749259 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f89c00e2770 con 0x7f89c81099c0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.050+0000 7f89cd200640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 0x7f89b00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.050+0000 7f89cdc00640 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 0x55569278a880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f89c8103350 tx=0x7f89bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff19f5e4068d29b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.050+0000 7f89baa00640 1 -- 172.21.15.50:0/1434121658 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc009d70 con 0x5556923bf2f0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.050+0000 7f89baa00640 1 -- 172.21.15.50:0/1434121658 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc005020 con 0x5556923bf2f0 2024-05-10T21:10:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.050+0000 7f89baa00640 1 -- 172.21.15.50:0/1434121658 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0049e0 con 0x5556923bf2f0 2024-05-10T21:10:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.051+0000 7f89d75c0740 1 -- 172.21.15.50:0/1434121658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 msgr2=0x55569278a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.051+0000 7f89d75c0740 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 0x55569278a880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f89c8103350 tx=0x7f89bc011810 comp rx=0 tx=0).stop 2024-05-10T21:10:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.051+0000 7f89cd200640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 0x7f89b00b4cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89b4032fd0 tx=0x7f89b4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 -- 172.21.15.50:0/1434121658 shutdown_connections 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923c74d0 0x555692791780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923bf2f0 0x55569278a880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 --2- 172.21.15.50:0/1434121658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923e81b0 0x55569278a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 -- 172.21.15.50:0/1434121658 >> 172.21.15.50:0/1434121658 conn(0x5556923dc300 msgr2=0x555692785c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 -- 172.21.15.50:0/1434121658 shutdown_connections 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 -- 172.21.15.50:0/1434121658 wait complete. 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.052+0000 7f89d75c0740 1 -- start start 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 Processor -- start 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692793df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5556927921d0 con 0x5556923e81b0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5556926a7660 con 0x5556923c74d0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.053+0000 7f89d75c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555692792350 con 0x5556923bf2f0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89d75c0740 10 client.0 _ll_register_callbacks cb 0x555692671400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cdc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692793df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48418/0 (socket says 172.21.15.50:48418) 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 -- 172.21.15.50:0/3563786381 learned_addr learned my addr 172.21.15.50:0/3563786381 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89ce600640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 -- 172.21.15.50:0/3563786381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 msgr2=0x555692793df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692793df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 -- 172.21.15.50:0/3563786381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 msgr2=0x55569279b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cd200640 1 -- 172.21.15.50:0/3563786381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556924cc620 con 0x5556923c74d0 2024-05-10T21:10:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.054+0000 7f89cdc00640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692793df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89cd200640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009230 tx=0x7f89b402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4227dbab04ef4ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89af400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4035040 con 0x5556923c74d0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc005020 con 0x5556923c74d0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f89b4046070 con 0x5556923c74d0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89b405c070 con 0x5556923c74d0 2024-05-10T21:10:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89d75c0740 10 client.25553 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.055+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55569279e2b0 con 0x5556923c74d0 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc005020 con 0x5556923c74d0 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f89b403d870 con 0x5556923c74d0 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 10 client.25553 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 20 client.25553 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5556923a6d80 0x5556927a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 4 client.25553 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556926fc160 con 0x5556923a6d80 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.056+0000 7f89cdc00640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5556923a6d80 0x5556927a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.057+0000 7f89cdc00640 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5556923a6d80 0x5556927a19f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25c3e53e292167d server_cookie=d30614a49c558dff in_seq=0 out_seq=0 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.057+0000 7f89af400640 10 client.25553 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.058+0000 7f89af400640 1 -- 172.21.15.50:0/3563786381 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556926fc160 con 0x5556923a6d80 2024-05-10T21:10:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.058+0000 7f89af400640 10 client.25553 handle_command_reply: tid=0 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.058+0000 7f89d75c0740 1 client.25553 shutdown 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.058+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 msgr2=0x555692794330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.058+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009230 tx=0x7f89b402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 shutdown_connections 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5556923a6d80 0x5556927a19f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692794330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89af400640 0 client.25553 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 --2- 172.21.15.50:0/3563786381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692793df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 >> 172.21.15.50:0/3563786381 conn(0x5556923dc300 msgr2=0x555692788ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 shutdown_connections 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 1 -- 172.21.15.50:0/3563786381 wait complete. 2024-05-10T21:10:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.059+0000 7f89d75c0740 20 client.25553 trim_cache size 0 max 16384 2024-05-10T21:10:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.083+0000 7f89d75c0740 1 Processor -- start 2024-05-10T21:10:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.083+0000 7f89d75c0740 1 -- start start 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.083+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692784250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.083+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x555692784cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89d75c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8010bf0 con 0x5556923e81b0 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89d75c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89a800e4c0 con 0x5556923c74d0 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89d75c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89a800c310 con 0x5556923bf2f0 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89ce600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x555692784cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89cdc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692784250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48434/0 (socket says 172.21.15.50:48434) 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.084+0000 7f89cd200640 1 -- 172.21.15.50:0/505185047 learned_addr learned my addr 172.21.15.50:0/505185047 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 -- 172.21.15.50:0/505185047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 msgr2=0x555692784250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692784250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 -- 172.21.15.50:0/505185047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 msgr2=0x555692784cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x555692784cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 -- 172.21.15.50:0/505185047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556927814a0 con 0x5556923c74d0 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89cd200640 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89b403dd10 tx=0x7f89b403df00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ef32b70818a713 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89ba000640 1 -- 172.21.15.50:0/505185047 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4066070 con 0x5556923c74d0 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89ba000640 1 -- 172.21.15.50:0/505185047 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4050070 con 0x5556923c74d0 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.085+0000 7f89ba000640 1 -- 172.21.15.50:0/505185047 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4035070 con 0x5556923c74d0 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.086+0000 7f89d75c0740 1 -- 172.21.15.50:0/505185047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 msgr2=0x555692784790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.086+0000 7f89d75c0740 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89b403dd10 tx=0x7f89b403df00 comp rx=0 tx=0).stop 2024-05-10T21:10:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.087+0000 7f89d75c0740 1 -- 172.21.15.50:0/505185047 shutdown_connections 2024-05-10T21:10:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.087+0000 7f89d75c0740 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x555692784cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.087+0000 7f89d75c0740 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x555692784790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.087+0000 7f89d75c0740 1 --2- 172.21.15.50:0/505185047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692784250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.087+0000 7f89d75c0740 1 -- 172.21.15.50:0/505185047 >> 172.21.15.50:0/505185047 conn(0x5556923dc300 msgr2=0x55569278e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.088+0000 7f89d75c0740 1 -- 172.21.15.50:0/505185047 shutdown_connections 2024-05-10T21:10:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.088+0000 7f89d75c0740 1 -- 172.21.15.50:0/505185047 wait complete. 2024-05-10T21:10:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.088+0000 7f89d75c0740 1 -- start start 2024-05-10T21:10:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 Processor -- start 2024-05-10T21:10:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692798bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89a800c980 con 0x5556923e81b0 2024-05-10T21:10:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89a800ddf0 con 0x5556923c74d0 2024-05-10T21:10:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f89a800d050 con 0x5556923bf2f0 2024-05-10T21:10:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.089+0000 7f89d75c0740 10 client.0 _ll_register_callbacks cb 0x7f89b4005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cdc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692798bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48448/0 (socket says 172.21.15.50:48448) 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 -- 172.21.15.50:0/920242695 learned_addr learned my addr 172.21.15.50:0/920242695 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89ce600640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 -- 172.21.15.50:0/920242695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 msgr2=0x555692798bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692798bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 -- 172.21.15.50:0/920242695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 msgr2=0x55569279a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89cd200640 1 -- 172.21.15.50:0/920242695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55569279a5e0 con 0x5556923c74d0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.090+0000 7f89ce600640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89cd200640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89b4003800 tx=0x7f89b402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3626fd899b2d2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89af400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4066070 con 0x5556923c74d0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b4050070 con 0x5556923c74d0 2024-05-10T21:10:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f89b403d650 con 0x5556923c74d0 2024-05-10T21:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89b4035070 con 0x5556923c74d0 2024-05-10T21:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89d75c0740 10 client.25559 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.091+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55569279b870 con 0x5556923c74d0 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4050070 con 0x5556923c74d0 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f89b4055470 con 0x5556923c74d0 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 10 client.25559 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 20 client.25559 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55569237c1c0 0x555692814790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 4 client.25559 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:10:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.092+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556926f5fd0 con 0x55569237c1c0 2024-05-10T21:10:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.093+0000 7f89cdc00640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55569237c1c0 0x555692814790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.093+0000 7f89cdc00640 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55569237c1c0 0x555692814790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100fa68126ba766a server_cookie=2aa744d3ec931055 in_seq=0 out_seq=0 2024-05-10T21:10:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.093+0000 7f89af400640 10 client.25559 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89af400640 1 -- 172.21.15.50:0/920242695 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556926f5fd0 con 0x55569237c1c0 2024-05-10T21:10:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89af400640 10 client.25559 handle_command_reply: tid=0 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 client.25559 shutdown 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 msgr2=0x5556927990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89b4003800 tx=0x7f89b402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 shutdown_connections 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55569237c1c0 0x555692814790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5556923e81b0 0x55569279a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556923c74d0 0x5556927990f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89af400640 0 client.25559 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 --2- 172.21.15.50:0/920242695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556923bf2f0 0x555692798bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 >> 172.21.15.50:0/920242695 conn(0x5556923dc300 msgr2=0x555692784250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 shutdown_connections 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 1 -- 172.21.15.50:0/920242695 wait complete. 2024-05-10T21:10:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.098+0000 7f89d75c0740 20 client.25559 trim_cache size 0 max 16384 2024-05-10T21:10:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:10:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:10:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:10:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:10:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 msgr2=0x7f89b00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 0x7f89b00b4cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89b4032fd0 tx=0x7f89b4031400 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 msgr2=0x7f89c8075560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f89c000eb10 tx=0x7f89c000c2d0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 shutdown_connections 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f89b00b27e0 0x7f89b00b4cf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89c81099c0 0x7f89c8075560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89c8104cc0 0x7f89c8075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 --2- 172.21.15.50:0/3095749259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c8103a80 0x7f89c8076980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.099+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 >> 172.21.15.50:0/3095749259 conn(0x7f89c80fec70 msgr2=0x7f89c81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.100+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 shutdown_connections 2024-05-10T21:10:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:07.100+0000 7f89cf000640 1 -- 172.21.15.50:0/3095749259 wait complete. 2024-05-10T21:10:07.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:06 smithi092 ceph-mon[34415]: pgmap v497: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:10:07.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:06 smithi142 ceph-mon[34846]: pgmap v497: 97 pgs: 97 active+clean; 672 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 -- 172.21.15.50:0/1773395049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb0105be0 msgr2=0x7f1cb010e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1773395049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb0105be0 0x7f1cb010e130 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009990 tx=0x7f1c9c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 -- 172.21.15.50:0/1773395049 shutdown_connections 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1773395049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105290 0x7f1cb01056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1773395049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb010e700 0x7f1cb0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1773395049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb0105be0 0x7f1cb010e130 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 -- 172.21.15.50:0/1773395049 >> 172.21.15.50:0/1773395049 conn(0x7f1cb0078ac0 msgr2=0x7f1cb0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 -- 172.21.15.50:0/1773395049 shutdown_connections 2024-05-10T21:10:07.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.579+0000 7f1cb5200700 1 -- 172.21.15.50:0/1773395049 wait complete. 2024-05-10T21:10:07.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.580+0000 7f1cb5200700 1 Processor -- start 2024-05-10T21:10:07.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 -- start start 2024-05-10T21:10:07.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 0x7f1cb01034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb010e700 0x7f1cb0103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01142b0 con 0x7f1cb010e700 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0114130 con 0x7f1cb0105be0 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1cb5200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0114430 con 0x7f1cb0105290 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1caf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.581+0000 7f1caf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:51020/0 (socket says 172.21.15.50:51020) 2024-05-10T21:10:07.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 -- 172.21.15.50:0/1766667186 learned_addr learned my addr 172.21.15.50:0/1766667186 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caea00700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 0x7f1cb01034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 -- 172.21.15.50:0/1766667186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 msgr2=0x7f1cb01034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 0x7f1cb01034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 -- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb010e700 msgr2=0x7f1cb0103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb010e700 0x7f1cb0103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 -- 172.21.15.50:0/1766667186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01a9040 con 0x7f1cb0105290 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caf400700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9a0 tx=0x7f1ca000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.582+0000 7f1caea00700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 0x7f1cb01034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:07.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.583+0000 7f1ca7e00700 1 -- 172.21.15.50:0/1766667186 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000cbd0 con 0x7f1cb0105290 2024-05-10T21:10:07.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.583+0000 7f1ca7e00700 1 -- 172.21.15.50:0/1766667186 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca000cd70 con 0x7f1cb0105290 2024-05-10T21:10:07.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.583+0000 7f1ca7e00700 1 -- 172.21.15.50:0/1766667186 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0018680 con 0x7f1cb0105290 2024-05-10T21:10:07.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.583+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01a9330 con 0x7f1cb0105290 2024-05-10T21:10:07.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.583+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a9820 con 0x7f1cb0105290 2024-05-10T21:10:07.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.587+0000 7f1ca7e00700 1 -- 172.21.15.50:0/1766667186 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ca0010430 con 0x7f1cb0105290 2024-05-10T21:10:07.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.589+0000 7f1ca7e00700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 0x7f1c940b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.589+0000 7f1ca7e00700 1 -- 172.21.15.50:0/1766667186 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1ca00e2470 con 0x7f1cb0105290 2024-05-10T21:10:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.590+0000 7f1caea00700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 0x7f1c940b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.590+0000 7f1cc5302b80 1 Processor -- start 2024-05-10T21:10:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.590+0000 7f1cc5302b80 1 -- start start 2024-05-10T21:10:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1caea00700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 0x7f1c940b4e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00b2e0 tx=0x7f1c9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0bd3aa0 con 0x55b2b0cb2300 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0b717e0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0bcfa60 con 0x55b2b0cb2710 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1caf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1caf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48474/0 (socket says 172.21.15.50:48474) 2024-05-10T21:10:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.591+0000 7f1caf400700 1 -- 172.21.15.50:0/3407769909 learned_addr learned my addr 172.21.15.50:0/3407769909 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 -- 172.21.15.50:0/3407769909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 msgr2=0x55b2b0cbdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caea00700 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 -- 172.21.15.50:0/3407769909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 msgr2=0x55b2b0cbb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 -- 172.21.15.50:0/3407769909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b0b20950 con 0x55b2b0b33ec0 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caea00700 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.592+0000 7f1caf400700 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ca008ffd0 tx=0x7f1ca00a92d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aba87f1ed7de816 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.593+0000 7f1ca4c00700 1 -- 172.21.15.50:0/3407769909 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00aa160 con 0x55b2b0b33ec0 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.593+0000 7f1ca4c00700 1 -- 172.21.15.50:0/3407769909 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00a5050 con 0x55b2b0b33ec0 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.593+0000 7f1ca4c00700 1 -- 172.21.15.50:0/3407769909 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00b2430 con 0x55b2b0b33ec0 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.593+0000 7f1cc5302b80 1 -- 172.21.15.50:0/3407769909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 msgr2=0x55b2b0cb1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.593+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ca008ffd0 tx=0x7f1ca00a92d0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 -- 172.21.15.50:0/3407769909 shutdown_connections 2024-05-10T21:10:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cb1dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/3407769909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 -- 172.21.15.50:0/3407769909 >> 172.21.15.50:0/3407769909 conn(0x55b2b0b72b00 msgr2=0x55b2b0cacbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 -- 172.21.15.50:0/3407769909 shutdown_connections 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.594+0000 7f1cc5302b80 1 -- 172.21.15.50:0/3407769909 wait complete. 2024-05-10T21:10:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.595+0000 7f1cc5302b80 1 -- start start 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.595+0000 7f1cc5302b80 1 Processor -- start 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0ccf830 con 0x55b2b0cb2300 2024-05-10T21:10:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0b39ba0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0ccf9b0 con 0x55b2b0cb2710 2024-05-10T21:10:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1cc5302b80 10 client.0 _ll_register_callbacks cb 0x55b2b0b67400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1caf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1caf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48486/0 (socket says 172.21.15.50:48486) 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.596+0000 7f1caf400700 1 -- 172.21.15.50:0/2337997287 learned_addr learned my addr 172.21.15.50:0/2337997287 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caea00700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caf400700 1 -- 172.21.15.50:0/2337997287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 msgr2=0x55b2b0cbbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caf400700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caf400700 1 -- 172.21.15.50:0/2337997287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 msgr2=0x55b2b0cbb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caf400700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caf400700 1 -- 172.21.15.50:0/2337997287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b0b994a0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.597+0000 7f1caea00700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1caf400700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c5b0 tx=0x7f1ca00a52b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821acc4152469ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1c99600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00c2030 con 0x55b2b0b33ec0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1ca00a6050 con 0x55b2b0b33ec0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca00a5050 con 0x55b2b0b33ec0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ca0085030 con 0x55b2b0b33ec0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1cc5302b80 10 client.25568 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.598+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2b0cda430 con 0x55b2b0b33ec0 2024-05-10T21:10:07.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.599+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00a5050 con 0x55b2b0b33ec0 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1ca00cc140 con 0x55b2b0b33ec0 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 20 client.25568 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0cdc900 0x55b2b0cded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 4 client.25568 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b0c7e0f0 con 0x55b2b0cdc900 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.600+0000 7f1caea00700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0cdc900 0x55b2b0cded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.601+0000 7f1caea00700 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0cdc900 0x55b2b0cded20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54d679ec469bc3f server_cookie=fe861ce2b39059d5 in_seq=0 out_seq=0 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.601+0000 7f1c99600700 10 client.25568 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.602+0000 7f1c99600700 1 -- 172.21.15.50:0/2337997287 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2b0c7e0f0 con 0x55b2b0cdc900 2024-05-10T21:10:07.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.602+0000 7f1c99600700 10 client.25568 handle_command_reply: tid=0 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.602+0000 7f1cc5302b80 1 client.25568 shutdown 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.602+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 msgr2=0x55b2b0cbb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c5b0 tx=0x7f1ca00a52b0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 shutdown_connections 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0cdc900 0x55b2b0cded20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbb420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cbbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/2337997287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 >> 172.21.15.50:0/2337997287 conn(0x55b2b0b72b00 msgr2=0x55b2b0cb0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1c99600700 0 client.25568 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 shutdown_connections 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 1 -- 172.21.15.50:0/2337997287 wait complete. 2024-05-10T21:10:07.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.603+0000 7f1cc5302b80 20 client.25568 trim_cache size 0 max 16384 2024-05-10T21:10:07.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.628+0000 7f1cc5302b80 1 Processor -- start 2024-05-10T21:10:07.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.628+0000 7f1cc5302b80 1 -- start start 2024-05-10T21:10:07.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.629+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.629+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.629+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cb6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.629+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01aad90 con 0x55b2b0b33ec0 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1caea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.629+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0074590 con 0x55b2b0cb2300 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1caea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48500/0 (socket says 172.21.15.50:48500) 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1caea00700 1 -- 172.21.15.50:0/145884169 learned_addr learned my addr 172.21.15.50:0/145884169 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0072400 con 0x55b2b0cb2710 2024-05-10T21:10:07.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1caf400700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1caea00700 1 -- 172.21.15.50:0/145884169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 msgr2=0x55b2b0cb6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.630+0000 7f1cafe00700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cb6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1caea00700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cb6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1caea00700 1 -- 172.21.15.50:0/145884169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 msgr2=0x55b2b0cbbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1caea00700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1caea00700 1 -- 172.21.15.50:0/145884169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b0cb13a0 con 0x55b2b0cb2300 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1cafe00700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.631+0000 7f1caf400700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.632+0000 7f1caea00700 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009990 tx=0x7f1c9c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f34779bde7dc9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.632+0000 7f1c9be00700 1 -- 172.21.15.50:0/145884169 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c036070 con 0x55b2b0cb2300 2024-05-10T21:10:07.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.632+0000 7f1c9be00700 1 -- 172.21.15.50:0/145884169 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9c00fcf0 con 0x55b2b0cb2300 2024-05-10T21:10:07.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.632+0000 7f1c9be00700 1 -- 172.21.15.50:0/145884169 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c023c80 con 0x55b2b0cb2300 2024-05-10T21:10:07.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 msgr2=0x55b2b0cb6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009990 tx=0x7f1c9c020000 comp rx=0 tx=0).stop 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 shutdown_connections 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0cb2710 0x55b2b0cb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cbbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/145884169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2300 0x55b2b0cb6720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.633+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 >> 172.21.15.50:0/145884169 conn(0x55b2b0b72b00 msgr2=0x55b2b0cabf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.634+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 shutdown_connections 2024-05-10T21:10:07.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.634+0000 7f1cc5302b80 1 -- 172.21.15.50:0/145884169 wait complete. 2024-05-10T21:10:07.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.634+0000 7f1cc5302b80 1 -- start start 2024-05-10T21:10:07.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 Processor -- start 2024-05-10T21:10:07.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cdfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2710 0x55b2b0d4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0072a50 con 0x55b2b0cb2300 2024-05-10T21:10:07.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0073ec0 con 0x55b2b0cb2710 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.635+0000 7f1cc5302b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0073120 con 0x55b2b0b33ec0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1cc5302b80 10 client.0 _ll_register_callbacks cb 0x55b2b0b67400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1caf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1caea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cdfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1caf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:51058/0 (socket says 172.21.15.50:51058) 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1caf400700 1 -- 172.21.15.50:0/4170252556 learned_addr learned my addr 172.21.15.50:0/4170252556 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.636+0000 7f1cafe00700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2710 0x55b2b0d4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 -- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2710 msgr2=0x55b2b0d4e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2710 0x55b2b0d4e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 -- 172.21.15.50:0/4170252556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 msgr2=0x55b2b0cdfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cdfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 -- 172.21.15.50:0/4170252556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b0ce0640 con 0x55b2b0b33ec0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1caf400700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00c42c0 tx=0x7f1ca000efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8eab99090d16023 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:07.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.637+0000 7f1c99600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00b28c0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f1ca00a6050 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c9c00fcf0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ca00a8ed0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1cc5302b80 10 client.25555 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.638+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2b0d4fb60 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9c00fcf0 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1ca00a8850 con 0x55b2b0b33ec0 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 10 client.25555 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 20 client.25555 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0d50020 0x55b2b0d52440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 4 client.25555 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.639+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2b0be1480 con 0x55b2b0d50020 2024-05-10T21:10:07.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.640+0000 7f1caea00700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0d50020 0x55b2b0d52440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:07.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.640+0000 7f1caea00700 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0d50020 0x55b2b0d52440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568e9dbcfff2af04 server_cookie=75a00831e475b55a in_seq=0 out_seq=0 2024-05-10T21:10:07.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.640+0000 7f1c99600700 10 client.25555 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.642+0000 7f1c99600700 1 -- 172.21.15.50:0/4170252556 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2b0be1480 con 0x55b2b0d50020 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.642+0000 7f1c99600700 10 client.25555 handle_command_reply: tid=0 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.642+0000 7f1cc5302b80 1 client.25555 shutdown 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 msgr2=0x55b2b0cdf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00c42c0 tx=0x7f1ca000efa0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 shutdown_connections 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b2b0d50020 0x55b2b0d52440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2b0cb2710 0x55b2b0d4e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2b0b33ec0 0x55b2b0cdf870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 --2- 172.21.15.50:0/4170252556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b2b0cb2300 0x55b2b0cdfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 >> 172.21.15.50:0/4170252556 conn(0x55b2b0b72b00 msgr2=0x55b2b0cac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.643+0000 7f1c99600700 0 client.25555 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.644+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 shutdown_connections 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.644+0000 7f1cc5302b80 1 -- 172.21.15.50:0/4170252556 wait complete. 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.644+0000 7f1cc5302b80 20 client.25555 trim_cache size 0 max 16384 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "b6f4feb0-fc8b-46dd-99c2-5f6efe0b1ce0", 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:10:07.644 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 msgr2=0x7f1c940b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 0x7f1c940b4e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00b2e0 tx=0x7f1c9c005fb0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 msgr2=0x7f1cb0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9a0 tx=0x7f1ca000ed60 comp rx=0 tx=0).stop 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 shutdown_connections 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1c940b2980 0x7f1c940b4e90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb010e700 0x7f1cb0103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb0105290 0x7f1cb0102f80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 --2- 172.21.15.50:0/1766667186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cb0105be0 0x7f1cb01034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.645+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 >> 172.21.15.50:0/1766667186 conn(0x7f1cb0078ac0 msgr2=0x7f1cb00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.646+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 shutdown_connections 2024-05-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:07.646+0000 7f1cb5200700 1 -- 172.21.15.50:0/1766667186 wait complete. 2024-05-10T21:10:09.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:08 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:10:09.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:10:09.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:08 smithi092 ceph-mon[34415]: pgmap v498: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T21:10:09.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:08 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:10:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:10:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:08 smithi142 ceph-mon[34846]: pgmap v498: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T21:10:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:08 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:10:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:10:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:08 smithi050 ceph-mon[29546]: pgmap v498: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T21:10:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:09 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:10:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:09 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:10:10.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:09 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:10:11.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:10 smithi092 ceph-mon[34415]: pgmap v499: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:11.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:10 smithi142 ceph-mon[34846]: pgmap v499: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:11.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:10 smithi050 ceph-mon[29546]: pgmap v499: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:10:11.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:10:11.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:10:11.886 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:13.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:12 smithi092 ceph-mon[34415]: pgmap v500: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:13.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:12 smithi142 ceph-mon[34846]: pgmap v500: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:13.132 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:12 smithi050 ceph-mon[29546]: pgmap v500: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:13.521 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:15.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:14 smithi092 ceph-mon[34415]: pgmap v501: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:15.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:14 smithi142 ceph-mon[34846]: pgmap v501: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:15.136 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:14 smithi050 ceph-mon[29546]: pgmap v501: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:15.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.955+0000 7f3153200700 1 -- 172.21.15.50:0/130176533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 msgr2=0x7f3154107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.955+0000 7f3153200700 1 --2- 172.21.15.50:0/130176533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f3154107f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 -- 172.21.15.50:0/130176533 shutdown_connections 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 --2- 172.21.15.50:0/130176533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31541084d0 0x7f31541089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 --2- 172.21.15.50:0/130176533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3154103080 0x7f3154103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 --2- 172.21.15.50:0/130176533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f3154107f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 -- 172.21.15.50:0/130176533 >> 172.21.15.50:0/130176533 conn(0x7f3154078ac0 msgr2=0x7f3154078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 -- 172.21.15.50:0/130176533 shutdown_connections 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.956+0000 7f3153200700 1 -- 172.21.15.50:0/130176533 wait complete. 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.957+0000 7f3153200700 1 Processor -- start 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.957+0000 7f3153200700 1 -- start start 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f315406e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31541084d0 0x7f315406eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f315411a860 con 0x7f31541084d0 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f315411a6e0 con 0x7f3154103080 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3153200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f315411a9e0 con 0x7f31541039d0 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3151e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f315406e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3151e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:35944/0 (socket says 172.21.15.50:35944) 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f315406e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41584/0 (socket says 172.21.15.50:41584) 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3151e00700 1 -- 172.21.15.50:0/2918966036 learned_addr learned my addr 172.21.15.50:0/2918966036 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.958+0000 7f3152800700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31541084d0 0x7f315406eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 -- 172.21.15.50:0/2918966036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 msgr2=0x7f315406e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f315406e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 -- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31541084d0 msgr2=0x7f315406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31541084d0 0x7f315406eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 -- 172.21.15.50:0/2918966036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3154072f00 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3151e00700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f314400eb30 tx=0x7f314400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f314aa00700 1 -- 172.21.15.50:0/2918966036 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314400cbd0 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.959+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31540731f0 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.960+0000 7f314aa00700 1 -- 172.21.15.50:0/2918966036 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31440044d0 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.960+0000 7f314aa00700 1 -- 172.21.15.50:0/2918966036 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144003e60 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.960+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3154073750 con 0x7f3154103080 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.964+0000 7f3167555b80 1 Processor -- start 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 -- start start 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096257d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558096179a20 con 0x558096258280 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558096117760 con 0x5580960d9e40 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.965+0000 7f3167555b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580961759e0 con 0x558096258690 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f314aa00700 1 -- 172.21.15.50:0/2918966036 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3144018430 con 0x7f3154103080 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41588/0 (socket says 172.21.15.50:41588) 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f3151400700 1 -- 172.21.15.50:0/3909300250 learned_addr learned my addr 172.21.15.50:0/3909300250 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f3152800700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096257d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.966+0000 7f3151e00700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151400700 1 -- 172.21.15.50:0/3909300250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 msgr2=0x558096257d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151400700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096257d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151400700 1 -- 172.21.15.50:0/3909300250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 msgr2=0x5580962618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151400700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151400700 1 -- 172.21.15.50:0/3909300250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580960c68d0 con 0x558096258690 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.967+0000 7f3151e00700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f3151400700 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7b6b4603938bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f314aa00700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 0x7f31380b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f3143400700 1 -- 172.21.15.50:0/3909300250 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c027070 con 0x558096258690 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f3143400700 1 -- 172.21.15.50:0/3909300250 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c0056b0 con 0x558096258690 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f3143400700 1 -- 172.21.15.50:0/3909300250 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c003c40 con 0x558096258690 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.968+0000 7f314aa00700 1 -- 172.21.15.50:0/2918966036 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f31440e24d0 con 0x7f3154103080 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.969+0000 7f3152800700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 0x7f31380b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 -- 172.21.15.50:0/3909300250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 msgr2=0x558096263e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c0096a0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3152800700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 0x7f31380b4e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x558096253500 tx=0x7f314c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 -- 172.21.15.50:0/3909300250 shutdown_connections 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096257d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258690 0x558096263e30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 --2- 172.21.15.50:0/3909300250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.970+0000 7f3167555b80 1 -- 172.21.15.50:0/3909300250 >> 172.21.15.50:0/3909300250 conn(0x558096118a80 msgr2=0x558096252b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.971+0000 7f3167555b80 1 -- 172.21.15.50:0/3909300250 shutdown_connections 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.971+0000 7f3167555b80 1 -- 172.21.15.50:0/3909300250 wait complete. 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.971+0000 7f3167555b80 1 -- start start 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 Processor -- start 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580960d9e40 0x5580962613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558096264740 con 0x558096258280 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580960dfb20 con 0x558096258690 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.972+0000 7f3167555b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580962648c0 con 0x5580960d9e40 2024-05-10T21:10:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.973+0000 7f3167555b80 10 client.0 _ll_register_callbacks cb 0x55809610d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.973+0000 7f3152800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.973+0000 7f3151e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580960d9e40 0x5580962613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.973+0000 7f3152800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:35964/0 (socket says 172.21.15.50:35964) 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.973+0000 7f3152800700 1 -- 172.21.15.50:0/3632593108 learned_addr learned my addr 172.21.15.50:0/3632593108 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 -- 172.21.15.50:0/3632593108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580960d9e40 msgr2=0x5580962613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580960d9e40 0x5580962613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3151400700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 -- 172.21.15.50:0/3632593108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 msgr2=0x5580962618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 -- 172.21.15.50:0/3632593108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55809613f420 con 0x558096258690 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3152800700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f314c016e00 tx=0x7f314c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb714e55a9da7aea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3137e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.974+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c026070 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.975+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f314c02b460 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.975+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f313c0056b0 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.975+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f314c02b660 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.975+0000 7f3167555b80 10 client.25586 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.975+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55809626f430 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.976+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c0056b0 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f314c02b800 con 0x558096258690 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 10 client.25586 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 20 client.25586 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558096271a70 0x558096273ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 4 client.25586 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558096224070 con 0x558096271a70 2024-05-10T21:10:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.977+0000 7f3151e00700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558096271a70 0x558096273ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.978+0000 7f3151e00700 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558096271a70 0x558096273ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85b4f0883d8fc40 server_cookie=e0928a43e7203a80 in_seq=0 out_seq=0 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.978+0000 7f3137e00700 10 client.25586 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3137e00700 1 -- 172.21.15.50:0/3632593108 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558096224070 con 0x558096271a70 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3137e00700 10 client.25586 handle_command_reply: tid=0 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 client.25586 shutdown 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 msgr2=0x558096261e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f314c016e00 tx=0x7f314c004b40 comp rx=0 tx=0).stop 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 shutdown_connections 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558096271a70 0x558096273ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096261e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580960d9e40 0x5580962613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 --2- 172.21.15.50:0/3632593108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258280 0x5580962618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 >> 172.21.15.50:0/3632593108 conn(0x558096118a80 msgr2=0x558096256700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3137e00700 0 client.25586 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 shutdown_connections 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.979+0000 7f3167555b80 1 -- 172.21.15.50:0/3632593108 wait complete. 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:14.980+0000 7f3167555b80 20 client.25586 trim_cache size 0 max 16384 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.001+0000 7f3167555b80 1 Processor -- start 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.001+0000 7f3167555b80 1 -- start start 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096256e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 0x55809625c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31541c0060 con 0x558096258690 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31541bd8f0 con 0x5580960d9e40 2024-05-10T21:10:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3167555b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31541bb760 con 0x558096258280 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.002+0000 7f3151400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41596/0 (socket says 172.21.15.50:41596) 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 -- 172.21.15.50:0/2566935798 learned_addr learned my addr 172.21.15.50:0/2566935798 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151e00700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096256e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3152800700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 0x55809625c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 -- 172.21.15.50:0/2566935798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 msgr2=0x558096256e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096256e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 -- 172.21.15.50:0/2566935798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 msgr2=0x55809625c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 0x55809625c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3151400700 1 -- 172.21.15.50:0/2566935798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558096254050 con 0x558096258280 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.003+0000 7f3152800700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 0x55809625c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.004+0000 7f3151400700 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f313c0098e0 tx=0x7f313c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633b5c4d8f6604ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.004+0000 7f3142a00700 1 -- 172.21.15.50:0/2566935798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c032070 con 0x558096258280 2024-05-10T21:10:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.004+0000 7f3142a00700 1 -- 172.21.15.50:0/2566935798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c00bdd0 con 0x558096258280 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.004+0000 7f3142a00700 1 -- 172.21.15.50:0/2566935798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c027040 con 0x558096258280 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 -- 172.21.15.50:0/2566935798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 msgr2=0x55809626d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f313c0098e0 tx=0x7f313c000bc0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 -- 172.21.15.50:0/2566935798 shutdown_connections 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580960d9e40 0x558096256e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558096258690 0x55809625c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 --2- 172.21.15.50:0/2566935798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x55809626d800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.005+0000 7f3167555b80 1 -- 172.21.15.50:0/2566935798 >> 172.21.15.50:0/2566935798 conn(0x558096118a80 msgr2=0x558096256700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.006+0000 7f3167555b80 1 -- 172.21.15.50:0/2566935798 shutdown_connections 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.006+0000 7f3167555b80 1 -- 172.21.15.50:0/2566935798 wait complete. 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.007+0000 7f3167555b80 1 -- start start 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.007+0000 7f3167555b80 1 Processor -- start 2024-05-10T21:10:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x558096265e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096266370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31541bbdb0 con 0x5580960d9e40 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31541bd220 con 0x558096258690 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31541bc480 con 0x558096258280 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3167555b80 10 client.0 _ll_register_callbacks cb 0x55809610d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3151e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3151e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43176/0 (socket says 172.21.15.50:43176) 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3151e00700 1 -- 172.21.15.50:0/2412225909 learned_addr learned my addr 172.21.15.50:0/2412225909 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.008+0000 7f3151400700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x558096265e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3152800700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096266370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 -- 172.21.15.50:0/2412225909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 msgr2=0x558096265e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x558096265e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 -- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 msgr2=0x558096266370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096266370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 -- 172.21.15.50:0/2412225909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580962db380 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151400700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x558096265e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3151e00700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f31440a97c0 tx=0x7f3144003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236cdb27719cf5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.009+0000 7f3137e00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440a9410 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f31440a60b0 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f313c00bdd0 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31440a5250 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3167555b80 10 client.15468 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5580962dc650 con 0x5580960d9e40 2024-05-10T21:10:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.010+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c00bdd0 con 0x5580960d9e40 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f31440e20b0 con 0x5580960d9e40 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 20 client.15468 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580962dcd10 0x5580962df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 4 client.15468 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558096187400 con 0x5580962dcd10 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.011+0000 7f3151400700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580962dcd10 0x5580962df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3151400700 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580962dcd10 0x5580962df130 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4781e9e82640a502 server_cookie=b4ff6c5706489cff in_seq=0 out_seq=0 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3137e00700 10 client.15468 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3137e00700 1 -- 172.21.15.50:0/2412225909 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558096187400 con 0x5580962dcd10 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3137e00700 10 client.15468 handle_command_reply: tid=0 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3167555b80 1 client.15468 shutdown 2024-05-10T21:10:15.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 msgr2=0x5580962658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f31440a97c0 tx=0x7f3144003900 comp rx=0 tx=0).stop 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 shutdown_connections 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.012+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580962dcd10 0x5580962df130 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3137e00700 0 client.15468 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558096258690 0x558096266370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580960d9e40 0x5580962658f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 --2- 172.21.15.50:0/2412225909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558096258280 0x558096265e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 >> 172.21.15.50:0/2412225909 conn(0x558096118a80 msgr2=0x5580962629e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 shutdown_connections 2024-05-10T21:10:15.296 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 1 -- 172.21.15.50:0/2412225909 wait complete. 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.013+0000 7f3167555b80 20 client.15468 trim_cache size 0 max 16384 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 msgr2=0x7f31380b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 0x7f31380b4e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x558096253500 tx=0x7f314c00b040 comp rx=0 tx=0).stop 2024-05-10T21:10:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 msgr2=0x7f31540753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f314400eb30 tx=0x7f314400ee40 comp rx=0 tx=0).stop 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 shutdown_connections 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f31380b2980 0x7f31380b4e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3154103080 0x7f31540753d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31541084d0 0x7f315406eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 --2- 172.21.15.50:0/2918966036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31541039d0 0x7f315406e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.014+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 >> 172.21.15.50:0/2918966036 conn(0x7f3154078ac0 msgr2=0x7f3154105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.015+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 shutdown_connections 2024-05-10T21:10:15.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:15.015+0000 7f3153200700 1 -- 172.21.15.50:0/2918966036 wait complete. 2024-05-10T21:10:16.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:10:16.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:10:16.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.690+0000 7f9eaf200700 1 -- 172.21.15.50:0/835202518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0104400 msgr2=0x7f9eb0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.690+0000 7f9eaf200700 1 --2- 172.21.15.50:0/835202518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0104400 0x7f9eb0104810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00099c0 tx=0x7f9ea0009cd0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.691+0000 7f9eaf200700 1 -- 172.21.15.50:0/835202518 shutdown_connections 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.691+0000 7f9eaf200700 1 --2- 172.21.15.50:0/835202518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0109680 0x7f9eb0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.691+0000 7f9eaf200700 1 --2- 172.21.15.50:0/835202518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0104400 0x7f9eb0104810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.691+0000 7f9eaf200700 1 --2- 172.21.15.50:0/835202518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0108d30 0x7f9eb0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.691+0000 7f9eaf200700 1 -- 172.21.15.50:0/835202518 >> 172.21.15.50:0/835202518 conn(0x7f9eb0079d10 msgr2=0x7f9eb007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:16.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.693+0000 7f9eaf200700 1 -- 172.21.15.50:0/835202518 shutdown_connections 2024-05-10T21:10:16.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.693+0000 7f9eaf200700 1 -- 172.21.15.50:0/835202518 wait complete. 2024-05-10T21:10:16.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.694+0000 7f9eaf200700 1 Processor -- start 2024-05-10T21:10:16.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.694+0000 7f9eaf200700 1 -- start start 2024-05-10T21:10:16.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 0x7f9eb01a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:16.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0108d30 0x7f9eb01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:16.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 0x7f9eb01a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:16.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0112150 con 0x7f9eb0104400 2024-05-10T21:10:16.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111fd0 con 0x7f9eb0108d30 2024-05-10T21:10:16.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eaf200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01122d0 con 0x7f9eb0109680 2024-05-10T21:10:16.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eade00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 0x7f9eb01a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eae800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 0x7f9eb01a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eae800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 0x7f9eb01a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41636/0 (socket says 172.21.15.50:41636) 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.695+0000 7f9eae800700 1 -- 172.21.15.50:0/4103938196 learned_addr learned my addr 172.21.15.50:0/4103938196 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9ead400700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0108d30 0x7f9eb01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9eade00700 1 -- 172.21.15.50:0/4103938196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 msgr2=0x7f9eb01a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9eade00700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 0x7f9eb01a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9eade00700 1 -- 172.21.15.50:0/4103938196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0108d30 msgr2=0x7f9eb01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9eade00700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0108d30 0x7f9eb01a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:16.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.696+0000 7f9eade00700 1 -- 172.21.15.50:0/4103938196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb01a6200 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.697+0000 7f9eade00700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 0x7f9eb01a1330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098e0 tx=0x7f9ea0005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.698+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0026070 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.698+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea0003c70 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.698+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb01a6490 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.698+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000fb10 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.698+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a6970 con 0x7f9eb0104400 2024-05-10T21:10:16.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.700+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb01058b0 con 0x7f9eb0104400 2024-05-10T21:10:16.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.705+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ea00037c0 con 0x7f9eb0104400 2024-05-10T21:10:16.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.708+0000 7f9ea6a00700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 0x7f9e940b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:16.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.708+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9ea00e2f50 con 0x7f9eb0104400 2024-05-10T21:10:16.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.709+0000 7f9ead400700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 0x7f9e940b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:16.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.710+0000 7f9ead400700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 0x7f9e940b4f50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9eb007a160 tx=0x7f9e9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:16.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:16.712+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ea0013070 con 0x7f9eb0104400 2024-05-10T21:10:16.726 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:16.863 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:16 smithi050 ceph-mon[29546]: pgmap v502: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:17.118 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:16 smithi092 ceph-mon[34415]: pgmap v502: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:16 smithi142 ceph-mon[34846]: pgmap v502: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:17.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.259+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eb01a2690 con 0x7f9eb0104400 2024-05-10T21:10:17.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.261+0000 7f9ea6a00700 1 -- 172.21.15.50:0/4103938196 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f9ea00aad50 con 0x7f9eb0104400 2024-05-10T21:10:17.260 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:10:17.260 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:10:17.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 msgr2=0x7f9e940b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:17.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 0x7f9e940b4f50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9eb007a160 tx=0x7f9e9c009260 comp rx=0 tx=0).stop 2024-05-10T21:10:17.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 msgr2=0x7f9eb01a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:17.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 0x7f9eb01a1330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098e0 tx=0x7f9ea0005330 comp rx=0 tx=0).stop 2024-05-10T21:10:17.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 shutdown_connections 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9e940b2a40 0x7f9e940b4f50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9eb0109680 0x7f9eb01a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9eb0104400 0x7f9eb01a1330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 --2- 172.21.15.50:0/4103938196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb0108d30 0x7f9eb01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.265+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 >> 172.21.15.50:0/4103938196 conn(0x7f9eb0079d10 msgr2=0x7f9eb01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.266+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 shutdown_connections 2024-05-10T21:10:17.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:17.266+0000 7f9eaf200700 1 -- 172.21.15.50:0/4103938196 wait complete. 2024-05-10T21:10:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:10:17.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:17 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/4103938196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:18.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:17 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/4103938196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:18.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:17 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/4103938196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:18.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id b63a33a6 --release --await' 2024-05-10T21:10:18.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db --set-id b63a33a6 --release --await 2024-05-10T21:10:19.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:18 smithi142 ceph-mon[34846]: pgmap v503: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:19.137 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:19.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:18 smithi050 ceph-mon[29546]: pgmap v503: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:19.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:18 smithi092 ceph-mon[34415]: pgmap v503: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:19.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 -- 172.21.15.50:0/853835289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc108d30 msgr2=0x7ff0fc109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:19.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/853835289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc108d30 0x7ff0fc109140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009920 tx=0x7ff0e8009c30 comp rx=0 tx=0).stop 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 -- 172.21.15.50:0/853835289 shutdown_connections 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/853835289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/853835289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc104400 0x7ff0fc104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/853835289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc108d30 0x7ff0fc109140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 -- 172.21.15.50:0/853835289 >> 172.21.15.50:0/853835289 conn(0x7ff0fc079d10 msgr2=0x7ff0fc07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 -- 172.21.15.50:0/853835289 shutdown_connections 2024-05-10T21:10:19.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.983+0000 7ff0fbe00700 1 -- 172.21.15.50:0/853835289 wait complete. 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.985+0000 7ff0fbe00700 1 Processor -- start 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.985+0000 7ff0fbe00700 1 -- start start 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 0x7ff0fc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 0x7ff0fc19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc114300 con 0x7ff0fc104400 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc114180 con 0x7ff0fc109680 2024-05-10T21:10:19.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.986+0000 7ff0fbe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc114480 con 0x7ff0fc108d30 2024-05-10T21:10:19.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.987+0000 7ff0faa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 0x7ff0fc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:19.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.987+0000 7ff0fb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:19.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.987+0000 7ff0fb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc1a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36034/0 (socket says 172.21.15.50:36034) 2024-05-10T21:10:19.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.987+0000 7ff0fb400700 1 -- 172.21.15.50:0/718710747 learned_addr learned my addr 172.21.15.50:0/718710747 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:19.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.987+0000 7ff0fa000700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 0x7ff0fc19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0faa00700 1 -- 172.21.15.50:0/718710747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 msgr2=0x7ff0fc19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0faa00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 0x7ff0fc19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0faa00700 1 -- 172.21.15.50:0/718710747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 msgr2=0x7ff0fc1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0faa00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0faa00700 1 -- 172.21.15.50:0/718710747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc1a1e60 con 0x7ff0fc104400 2024-05-10T21:10:19.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.989+0000 7ff0fa000700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 0x7ff0fc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:19.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.990+0000 7ff0faa00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 0x7ff0fc19cf90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005fd0 tx=0x7ff0ec00c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.991+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec023070 con 0x7ff0fc104400 2024-05-10T21:10:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.991+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ec015d50 con 0x7ff0fc104400 2024-05-10T21:10:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.991+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec016340 con 0x7ff0fc104400 2024-05-10T21:10:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.991+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc1a2150 con 0x7ff0fc104400 2024-05-10T21:10:19.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.991+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc10af90 con 0x7ff0fc104400 2024-05-10T21:10:19.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.992+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc1058b0 con 0x7ff0fc104400 2024-05-10T21:10:19.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:19.997+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff0ec026070 con 0x7ff0fc104400 2024-05-10T21:10:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.000+0000 7ff100a00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 0x7ff0e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:19.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.000+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff0ec0ea3b0 con 0x7ff0fc104400 2024-05-10T21:10:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.001+0000 7ff0fa000700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 0x7ff0e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.002+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0ec0b20a0 con 0x7ff0fc104400 2024-05-10T21:10:20.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.002+0000 7ff0fa000700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 0x7ff0e00b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009810 tx=0x7ff0e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.319+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0fc10b2b0 con 0x7ff0fc104400 2024-05-10T21:10:20.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.320+0000 7ff100a00700 1 -- 172.21.15.50:0/718710747 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7ff0ec0b19a0 con 0x7ff0fc104400 2024-05-10T21:10:20.319 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:10:20.320 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:10:20.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 msgr2=0x7ff0e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 0x7ff0e00b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009810 tx=0x7ff0e8019040 comp rx=0 tx=0).stop 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 msgr2=0x7ff0fc19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 0x7ff0fc19cf90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005fd0 tx=0x7ff0ec00c630 comp rx=0 tx=0).stop 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 shutdown_connections 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff0e00b2900 0x7ff0e00b4e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0fc109680 0x7ff0fc1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0fc104400 0x7ff0fc19cf90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 --2- 172.21.15.50:0/718710747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0fc108d30 0x7ff0fc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.323+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 >> 172.21.15.50:0/718710747 conn(0x7ff0fc079d10 msgr2=0x7ff0fc10f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.324+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 shutdown_connections 2024-05-10T21:10:20.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:20.324+0000 7ff0fbe00700 1 -- 172.21.15.50:0/718710747 wait complete. 2024-05-10T21:10:20.326 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:10:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2297391230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 msgr2=0x7fa9c410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2297391230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c410bfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009be0 tx=0x7fa9bc02a6c0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2297391230 shutdown_connections 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2297391230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c410bfd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2297391230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4104eb0 0x7fa9c41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2297391230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4103c70 0x7fa9c4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.421+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2297391230 >> 172.21.15.50:0/2297391230 conn(0x7fa9c40fee60 msgr2=0x7fa9c4101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.422+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2297391230 shutdown_connections 2024-05-10T21:10:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.422+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2297391230 wait complete. 2024-05-10T21:10:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.423+0000 7fa9c9c00640 1 Processor -- start 2024-05-10T21:10:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.423+0000 7fa9c9c00640 1 -- start start 2024-05-10T21:10:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 0x7fa9c4077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 0x7fa9c4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c4076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f780 con 0x7fa9c4109bb0 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f600 con 0x7fa9c4103c70 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c9c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f900 con 0x7fa9c4104eb0 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c4076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 0x7fa9c4077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 0x7fa9c4077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36046/0 (socket says 172.21.15.50:36046) 2024-05-10T21:10:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.424+0000 7fa9c2a00640 1 -- 172.21.15.50:0/2235494072 learned_addr learned my addr 172.21.15.50:0/2235494072 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 0x7fa9c4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 -- 172.21.15.50:0/2235494072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 msgr2=0x7fa9c4077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 0x7fa9c4077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 -- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 msgr2=0x7fa9c4076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c4076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 -- 172.21.15.50:0/2235494072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c4076da0 con 0x7fa9c4104eb0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c3400640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c4076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.425+0000 7fa9c2000640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 0x7fa9c4076130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e790 tx=0x7fa9b000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.426+0000 7fa9bb400640 1 -- 172.21.15.50:0/2235494072 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000eea0 con 0x7fa9c4104eb0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.426+0000 7fa9bb400640 1 -- 172.21.15.50:0/2235494072 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b000ccc0 con 0x7fa9c4104eb0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.426+0000 7fa9bb400640 1 -- 172.21.15.50:0/2235494072 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0018660 con 0x7fa9c4104eb0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.426+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41a43b0 con 0x7fa9c4104eb0 2024-05-10T21:10:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.426+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a4820 con 0x7fa9c4104eb0 2024-05-10T21:10:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.430+0000 7fa9bb400640 1 -- 172.21.15.50:0/2235494072 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa9b0010430 con 0x7fa9c4104eb0 2024-05-10T21:10:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.431+0000 7fa9d0b6a740 1 Processor -- start 2024-05-10T21:10:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.431+0000 7fa9d0b6a740 1 -- start start 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872603910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 0x558872603e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 0x55887260ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55887255f550 con 0x5588722302f0 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558872504bc0 con 0x5588722384d0 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55887255b510 con 0x5588722591b0 2024-05-10T21:10:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9bb400640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 0x7fa9a80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9bb400640 1 -- 172.21.15.50:0/2235494072 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa9b00e2410 con 0x7fa9c4104eb0 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 0x558872603e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872603910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.432+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872603910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41684/0 (socket says 172.21.15.50:41684) 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c3400640 1 -- 172.21.15.50:0/1736398809 learned_addr learned my addr 172.21.15.50:0/1736398809 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c2000640 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 0x55887260ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c3400640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 0x7fa9a80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c2000640 1 -- 172.21.15.50:0/1736398809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 msgr2=0x558872603910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c2000640 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872603910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.433+0000 7fa9c2000640 1 -- 172.21.15.50:0/1736398809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 msgr2=0x558872603e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9c2000640 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 0x558872603e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9c2000640 1 -- 172.21.15.50:0/1736398809 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558872271210 con 0x5588722384d0 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 0x558872603e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9c3400640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 0x7fa9a80b4cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0099a0 tx=0x7fa9bc030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9c2000640 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 0x55887260ad90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00dfee0 tx=0x7fa9b00a5250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e943d93c25ca40c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9afe00640 1 -- 172.21.15.50:0/1736398809 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00b21b0 con 0x5588722384d0 2024-05-10T21:10:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9afe00640 1 -- 172.21.15.50:0/1736398809 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00a9a00 con 0x5588722384d0 2024-05-10T21:10:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.434+0000 7fa9afe00640 1 -- 172.21.15.50:0/1736398809 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0010720 con 0x5588722384d0 2024-05-10T21:10:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.435+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/1736398809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 msgr2=0x55887260ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.435+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 0x55887260ad90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00dfee0 tx=0x7fa9b00a5250 comp rx=0 tx=0).stop 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/1736398809 shutdown_connections 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722384d0 0x55887260ad90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722302f0 0x558872603e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/1736398809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872603910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/1736398809 >> 172.21.15.50:0/1736398809 conn(0x55887224d300 msgr2=0x5588725ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/1736398809 shutdown_connections 2024-05-10T21:10:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.436+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/1736398809 wait complete. 2024-05-10T21:10:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.437+0000 7fa9d0b6a740 1 -- start start 2024-05-10T21:10:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.437+0000 7fa9d0b6a740 1 Processor -- start 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x55887260d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872614870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55887260b720 con 0x5588722591b0 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558872518660 con 0x5588722302f0 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55887260b8a0 con 0x5588722384d0 2024-05-10T21:10:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9d0b6a740 10 client.0 _ll_register_callbacks cb 0x5588724e2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36056/0 (socket says 172.21.15.50:36056) 2024-05-10T21:10:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9c2a00640 1 -- 172.21.15.50:0/572736031 learned_addr learned my addr 172.21.15.50:0/572736031 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.438+0000 7fa9c2000640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x55887260d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 -- 172.21.15.50:0/572736031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 msgr2=0x55887260d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x55887260d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 -- 172.21.15.50:0/572736031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 msgr2=0x558872614870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c3400640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872614870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872614870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 -- 172.21.15.50:0/572736031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887233d620 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c3400640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872614870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bf10 tx=0x7fa9b4012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a41004559f00e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.439+0000 7fa9acc00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4005220 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00a9a00 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa9b40053c0 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9b40055c0 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9d0b6a740 10 client.25613 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.440+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588726178c0 con 0x5588722302f0 2024-05-10T21:10:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00a9a00 con 0x5588722302f0 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa9b4019070 con 0x5588722302f0 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 10 client.25613 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 20 client.25613 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558872217d80 0x55887261af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 4 client.25613 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.441+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55887256d160 con 0x558872217d80 2024-05-10T21:10:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.442+0000 7fa9c2000640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558872217d80 0x55887261af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.442+0000 7fa9c2000640 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558872217d80 0x55887261af90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89fd78d29858891 server_cookie=a72047fa542c74e2 in_seq=0 out_seq=0 2024-05-10T21:10:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.442+0000 7fa9acc00640 10 client.25613 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9acc00640 1 -- 172.21.15.50:0/572736031 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55887256d160 con 0x558872217d80 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9acc00640 10 client.25613 handle_command_reply: tid=0 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 client.25613 shutdown 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 msgr2=0x55887260d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bf10 tx=0x7fa9b4012820 comp rx=0 tx=0).stop 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 shutdown_connections 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558872217d80 0x55887261af90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x55887260d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872614870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/572736031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x55887260d3d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9acc00640 0 client.25613 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 >> 172.21.15.50:0/572736031 conn(0x55887224d300 msgr2=0x5588725f5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.443+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 shutdown_connections 2024-05-10T21:10:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.444+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/572736031 wait complete. 2024-05-10T21:10:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.444+0000 7fa9d0b6a740 20 client.25613 trim_cache size 0 max 16384 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 Processor -- start 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 -- start start 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722384d0 0x558872614790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41aba00 con 0x5588722384d0 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a92d0 con 0x5588722302f0 2024-05-10T21:10:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.460+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a7120 con 0x5588722591b0 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36068/0 (socket says 172.21.15.50:36068) 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 -- 172.21.15.50:0/3656794443 learned_addr learned my addr 172.21.15.50:0/3656794443 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41704/0 (socket says 172.21.15.50:41704) 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2000640 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722384d0 0x558872614790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 -- 172.21.15.50:0/3656794443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 msgr2=0x558872614cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 -- 172.21.15.50:0/3656794443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722384d0 msgr2=0x558872614790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722384d0 0x558872614790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c2a00640 1 -- 172.21.15.50:0/3656794443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588725f2280 con 0x5588722302f0 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.461+0000 7fa9c3400640 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.462+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4010d70 tx=0x7fa9b40059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c82ddc875537d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.462+0000 7fa9af400640 1 -- 172.21.15.50:0/3656794443 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4010960 con 0x5588722302f0 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.462+0000 7fa9af400640 1 -- 172.21.15.50:0/3656794443 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b40258d0 con 0x5588722302f0 2024-05-10T21:10:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.462+0000 7fa9af400640 1 -- 172.21.15.50:0/3656794443 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b401a700 con 0x5588722302f0 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/3656794443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 msgr2=0x5588725ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4010d70 tx=0x7fa9b40059b0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/3656794443 shutdown_connections 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722591b0 0x558872614cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722384d0 0x558872614790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/3656794443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588725ff240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/3656794443 >> 172.21.15.50:0/3656794443 conn(0x55887224d300 msgr2=0x5588725feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/3656794443 shutdown_connections 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.463+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/3656794443 wait complete. 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 -- start start 2024-05-10T21:10:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 Processor -- start 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588726120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x558872612620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a7790 con 0x5588722591b0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a8c00 con 0x5588722302f0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a7e60 con 0x5588722384d0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.464+0000 7fa9d0b6a740 10 client.0 _ll_register_callbacks cb 0x7fa9b400b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x558872612620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43266/0 (socket says 172.21.15.50:43266) 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c3400640 1 -- 172.21.15.50:0/31750152 learned_addr learned my addr 172.21.15.50:0/31750152 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588726120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 -- 172.21.15.50:0/31750152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 msgr2=0x558872612620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x558872612620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 -- 172.21.15.50:0/31750152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 msgr2=0x558872681330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2a00640 1 -- 172.21.15.50:0/31750152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588726818a0 con 0x5588722302f0 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c2000640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x558872612620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.465+0000 7fa9c3400640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9c2a00640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588726120e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5588725f2160 tx=0x7fa9b4025650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9217008a002cf49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9acc00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4032040 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa9b4025c80 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b40258d0 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9b403d070 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9d0b6a740 10 client.25619 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.466+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558872682af0 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b40258d0 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa9b4035070 con 0x5588722302f0 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 10 client.25619 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 20 client.25619 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5588721ed1c0 0x5588726854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 4 client.25619 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:10:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558872566fd0 con 0x5588721ed1c0 2024-05-10T21:10:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.467+0000 7fa9c2000640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5588721ed1c0 0x5588726854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.468+0000 7fa9c2000640 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5588721ed1c0 0x5588726854b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d6288a66aa035e server_cookie=f95b6229d7a9f1c in_seq=0 out_seq=0 2024-05-10T21:10:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.468+0000 7fa9acc00640 10 client.25619 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.472+0000 7fa9acc00640 1 -- 172.21.15.50:0/31750152 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558872566fd0 con 0x5588721ed1c0 2024-05-10T21:10:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.472+0000 7fa9acc00640 10 client.25619 handle_command_reply: tid=0 2024-05-10T21:10:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.472+0000 7fa9d0b6a740 1 client.25619 shutdown 2024-05-10T21:10:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.472+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 msgr2=0x5588726120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.472+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588726120e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5588725f2160 tx=0x7fa9b4025650 comp rx=0 tx=0).stop 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 shutdown_connections 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5588721ed1c0 0x5588726854b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588722384d0 0x558872612620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5588722591b0 0x558872681330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 --2- 172.21.15.50:0/31750152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588722302f0 0x5588726120e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9acc00640 0 client.25619 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 >> 172.21.15.50:0/31750152 conn(0x55887224d300 msgr2=0x55887260cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 shutdown_connections 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 1 -- 172.21.15.50:0/31750152 wait complete. 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.473+0000 7fa9d0b6a740 20 client.25619 trim_cache size 0 max 16384 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:10:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 msgr2=0x7fa9a80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 0x7fa9a80b4cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0099a0 tx=0x7fa9bc030970 comp rx=0 tx=0).stop 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 msgr2=0x7fa9c4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 0x7fa9c4076130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e790 tx=0x7fa9b000ec50 comp rx=0 tx=0).stop 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 shutdown_connections 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa9a80b27e0 0x7fa9a80b4cf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c4104eb0 0x7fa9c4076130 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9c4109bb0 0x7fa9c4076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 --2- 172.21.15.50:0/2235494072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9c4103c70 0x7fa9c4077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 >> 172.21.15.50:0/2235494072 conn(0x7fa9c40fee60 msgr2=0x7fa9c4105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 shutdown_connections 2024-05-10T21:10:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:20.474+0000 7fa9c9c00640 1 -- 172.21.15.50:0/2235494072 wait complete. 2024-05-10T21:10:21.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:20 smithi142 ceph-mon[34846]: pgmap v504: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T21:10:21.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/718710747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:21.129 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:20 smithi050 ceph-mon[29546]: pgmap v504: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T21:10:21.129 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/718710747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:21.189 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:10:21.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:20 smithi092 ceph-mon[34415]: pgmap v504: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T21:10:21.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/718710747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:21.694 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:22.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:22 smithi050 ceph-mon[29546]: pgmap v505: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T21:10:23.170 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:22 smithi092 ceph-mon[34415]: pgmap v505: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T21:10:23.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 -- 172.21.15.50:0/1474885157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac108f10 msgr2=0x7ff7ac109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 --2- 172.21.15.50:0/1474885157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac108f10 0x7ff7ac109320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7980099c0 tx=0x7ff798009cd0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 -- 172.21.15.50:0/1474885157 shutdown_connections 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 --2- 172.21.15.50:0/1474885157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac109860 0x7ff7ac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 --2- 172.21.15.50:0/1474885157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac104630 0x7ff7ac104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 --2- 172.21.15.50:0/1474885157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac108f10 0x7ff7ac109320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.192+0000 7ff7b3200700 1 -- 172.21.15.50:0/1474885157 >> 172.21.15.50:0/1474885157 conn(0x7ff7ac1000a0 msgr2=0x7ff7ac1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.193+0000 7ff7b3200700 1 -- 172.21.15.50:0/1474885157 shutdown_connections 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.193+0000 7ff7b3200700 1 -- 172.21.15.50:0/1474885157 wait complete. 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.195+0000 7ff7b3200700 1 Processor -- start 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 -- start start 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 0x7ff7ac10b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 0x7ff7ac10b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac109860 0x7ff7ac10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1144c0 con 0x7ff7ac109860 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac114340 con 0x7ff7ac104630 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.196+0000 7ff7b3200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac114640 con 0x7ff7ac108f10 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.197+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 0x7ff7ac10b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.197+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 0x7ff7ac10b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41746/0 (socket says 172.21.15.50:41746) 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.197+0000 7ff7ab400700 1 -- 172.21.15.50:0/2123520316 learned_addr learned my addr 172.21.15.50:0/2123520316 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.197+0000 7ff7abe00700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 0x7ff7ac10b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 -- 172.21.15.50:0/2123520316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 msgr2=0x7ff7ac10b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 0x7ff7ac10b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 -- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac109860 msgr2=0x7ff7ac10bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac109860 0x7ff7ac10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 -- 172.21.15.50:0/2123520316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac1ad630 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.198+0000 7ff7abe00700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 0x7ff7ac10b240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.199+0000 7ff7a8c00700 1 -- 172.21.15.50:0/2123520316 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00cb70 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.199+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac1ad8c0 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.199+0000 7ff7a8c00700 1 -- 172.21.15.50:0/2123520316 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c0044d0 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.199+0000 7ff7a8c00700 1 -- 172.21.15.50:0/2123520316 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c003e00 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.199+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac1ade50 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.205+0000 7ff7a8c00700 1 -- 172.21.15.50:0/2123520316 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff79c010490 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.207+0000 7ff7c1e2db80 1 Processor -- start 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.207+0000 7ff7a8c00700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 0x7ff7900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.208+0000 7ff7a8c00700 1 -- 172.21.15.50:0/2123520316 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff79c0e2310 con 0x7ff7ac104630 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.208+0000 7ff7c1e2db80 1 -- start start 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.209+0000 7ff7b1200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 0x7ff7900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.209+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff51c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56059fe73b20 con 0x56059fdd3f40 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56059fe11860 con 0x56059ff56920 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56059fe6fae0 con 0x56059ff52140 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7b1200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7b1200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36110/0 (socket says 172.21.15.50:36110) 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7b1200700 1 -- 172.21.15.50:0/1513396638 learned_addr learned my addr 172.21.15.50:0/1513396638 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.210+0000 7ff7b1200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 0x7ff7900b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e2d0 tx=0x7ff7a400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.211+0000 7ff7b1200700 1 -- 172.21.15.50:0/1513396638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 msgr2=0x56059ff563e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.211+0000 7ff7abe00700 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff51c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.211+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff563e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.211+0000 7ff7b1200700 1 -- 172.21.15.50:0/1513396638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 msgr2=0x56059ff51c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.211+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff51c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7b1200700 1 -- 172.21.15.50:0/1513396638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059fdc09d0 con 0x56059ff56920 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7abe00700 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff51c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4017440 tx=0x7ff7a4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13a7797861bcadd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7a1600700 1 -- 172.21.15.50:0/1513396638 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a401b040 con 0x56059ff56920 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7a1600700 1 -- 172.21.15.50:0/1513396638 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a4026070 con 0x56059ff56920 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.212+0000 7ff7a1600700 1 -- 172.21.15.50:0/1513396638 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a401f690 con 0x56059ff56920 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1513396638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 msgr2=0x56059ff56d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4017440 tx=0x7ff7a4019000 comp rx=0 tx=0).stop 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1513396638 shutdown_connections 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff56d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff51c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1513396638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1513396638 >> 172.21.15.50:0/1513396638 conn(0x56059fe19070 msgr2=0x56059ff4c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1513396638 shutdown_connections 2024-05-10T21:10:23.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.214+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1513396638 wait complete. 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.215+0000 7ff7c1e2db80 1 -- start start 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.215+0000 7ff7c1e2db80 1 Processor -- start 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56059ff5e810 con 0x56059fdd3f40 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56059fdd9c20 con 0x56059ff56920 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56059ff5e990 con 0x56059ff52140 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7c1e2db80 10 client.0 _ll_register_callbacks cb 0x56059fe1ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.216+0000 7ff7abe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36122/0 (socket says 172.21.15.50:36122) 2024-05-10T21:10:23.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 -- 172.21.15.50:0/1522172596 learned_addr learned my addr 172.21.15.50:0/1522172596 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7ab400700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 -- 172.21.15.50:0/1522172596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 msgr2=0x56059ff5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 -- 172.21.15.50:0/1522172596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 msgr2=0x56059ff5b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff5b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7b1200700 1 -- 172.21.15.50:0/1522172596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059fe39520 con 0x56059ff56920 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.217+0000 7ff7abe00700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.218+0000 7ff7b1200700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004b10 tx=0x7ff7a401f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7455b4857c4f394 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.218+0000 7ff796000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.218+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4030070 con 0x56059ff56920 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.218+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff7a4004bb0 con 0x56059ff56920 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.218+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a4026070 con 0x56059ff56920 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.219+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7a401b040 con 0x56059ff56920 2024-05-10T21:10:23.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.219+0000 7ff7c1e2db80 10 client.25631 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.219+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059ff69460 con 0x56059ff56920 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.219+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a4026070 con 0x56059ff56920 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff7a4033020 con 0x56059ff56920 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 10 client.25631 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 10 client.25631 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 20 client.25631 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ff6ba40 0x56059ff6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 4 client.25631 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059ff1e2c0 con 0x56059ff6ba40 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.220+0000 7ff7abe00700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ff6ba40 0x56059ff6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.221+0000 7ff7abe00700 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ff6ba40 0x56059ff6de60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0d7998f72244f server_cookie=2b128d59f1b2997f in_seq=0 out_seq=0 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.221+0000 7ff796000700 10 client.25631 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff796000700 1 -- 172.21.15.50:0/1522172596 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56059ff1e2c0 con 0x56059ff6ba40 2024-05-10T21:10:23.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff796000700 10 client.25631 handle_command_reply: tid=0 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 client.25631 shutdown 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 msgr2=0x56059ff5c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004b10 tx=0x7ff7a401f840 comp rx=0 tx=0).stop 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 shutdown_connections 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ff6ba40 0x56059ff6de60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff56920 0x56059ff5c120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/1522172596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.222+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 >> 172.21.15.50:0/1522172596 conn(0x56059fe19070 msgr2=0x56059ff50950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.223+0000 7ff796000700 0 client.25631 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.223+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 shutdown_connections 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.223+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/1522172596 wait complete. 2024-05-10T21:10:23.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.223+0000 7ff7c1e2db80 20 client.25631 trim_cache size 0 max 16384 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.265+0000 7ff7c1e2db80 1 Processor -- start 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.265+0000 7ff7c1e2db80 1 -- start start 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.265+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 0x56059ff5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 0x56059ff51060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b7b70 con 0x56059ff56920 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b5400 con 0x56059fdd3f40 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b3290 con 0x56059ff52140 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7abe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 0x56059ff5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7abe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 0x56059ff5c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36126/0 (socket says 172.21.15.50:36126) 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.266+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.267+0000 7ff7abe00700 1 -- 172.21.15.50:0/2702727656 learned_addr learned my addr 172.21.15.50:0/2702727656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.267+0000 7ff7b1200700 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 0x56059ff51060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.267+0000 7ff7ab400700 1 -- 172.21.15.50:0/2702727656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 msgr2=0x56059ff5c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.267+0000 7ff7ab400700 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 0x56059ff5c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.267+0000 7ff7ab400700 1 -- 172.21.15.50:0/2702727656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 msgr2=0x56059ff51060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.268+0000 7ff7ab400700 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 0x56059ff51060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.268+0000 7ff7ab400700 1 -- 172.21.15.50:0/2702727656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059ff4d440 con 0x56059ff52140 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.268+0000 7ff7b1200700 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 0x56059ff51060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.268+0000 7ff7ab400700 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff562a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac110440 tx=0x7ff798005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa9106fd04823848 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.271+0000 7ff7a0c00700 1 -- 172.21.15.50:0/2702727656 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980052b0 con 0x56059ff52140 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.271+0000 7ff7a0c00700 1 -- 172.21.15.50:0/2702727656 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff798028070 con 0x56059ff52140 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.271+0000 7ff7a0c00700 1 -- 172.21.15.50:0/2702727656 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79800b9d0 con 0x56059ff52140 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/2702727656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 msgr2=0x56059ff562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff562a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac110440 tx=0x7ff798005040 comp rx=0 tx=0).stop 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/2702727656 shutdown_connections 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059fdd3f40 0x56059ff5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059ff56920 0x56059ff51060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/2702727656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff52140 0x56059ff562a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/2702727656 >> 172.21.15.50:0/2702727656 conn(0x56059fe19070 msgr2=0x56059ff50950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/2702727656 shutdown_connections 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.273+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/2702727656 wait complete. 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.274+0000 7ff7c1e2db80 1 -- start start 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.274+0000 7ff7c1e2db80 1 Processor -- start 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 0x56059ff60980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 0x56059ffdf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b38e0 con 0x56059fdd3f40 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b4d30 con 0x56059ff52140 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1b3f90 con 0x56059ff56920 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7c1e2db80 10 client.0 _ll_register_callbacks cb 0x56059fe1ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:23.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7abe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 0x56059ff60980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 0x56059ff60980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36128/0 (socket says 172.21.15.50:36128) 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.275+0000 7ff7ab400700 1 -- 172.21.15.50:0/12129828 learned_addr learned my addr 172.21.15.50:0/12129828 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 0x56059ffdf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 -- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 msgr2=0x56059ff60980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 0x56059ff60980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 -- 172.21.15.50:0/12129828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 msgr2=0x56059ff60440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff60440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 -- 172.21.15.50:0/12129828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059ff4e290 con 0x56059ff56920 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.276+0000 7ff7b1200700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 0x56059ffdf080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4017ae0 tx=0x7ff7a401f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ae3da90debf7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.277+0000 7ff796000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.277+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4030070 con 0x56059ff56920 2024-05-10T21:10:23.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.277+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ff7a4021440 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.277+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff798028070 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.277+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7a4037450 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 10 client.25618 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059ffe7920 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff798028070 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff7a4033040 con 0x56059ff56920 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 10 client.25618 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 10 client.25618 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 20 client.25618 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ffe7ef0 0x56059ffea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 4 client.25618 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b63a33a6", "release": true, "await": true}] 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.278+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b63a33a6", "release": true, "await": true}) -- 0x56059fe81500 con 0x56059ffe7ef0 2024-05-10T21:10:23.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.279+0000 7ff7abe00700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ffe7ef0 0x56059ffea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.279+0000 7ff7abe00700 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ffe7ef0 0x56059ffea310 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4dc05137e40d37 server_cookie=864baed30c58d74a in_seq=0 out_seq=0 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.279+0000 7ff796000700 10 client.25618 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.281+0000 7ff796000700 1 -- 172.21.15.50:0/12129828 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56059fe81500 con 0x56059ffe7ef0 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.281+0000 7ff796000700 10 client.25618 handle_command_reply: tid=0 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.281+0000 7ff7c1e2db80 1 client.25618 shutdown 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.281+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 msgr2=0x56059ffdf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.281+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 0x56059ffdf080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4017ae0 tx=0x7ff7a401f460 comp rx=0 tx=0).stop 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 shutdown_connections 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56059ffe7ef0 0x56059ffea310 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56059ff56920 0x56059ffdf080 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56059fdd3f40 0x56059ff60440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 --2- 172.21.15.50:0/12129828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56059ff52140 0x56059ff60980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 >> 172.21.15.50:0/12129828 conn(0x56059fe19070 msgr2=0x56059ff591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff796000700 0 client.25618 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 shutdown_connections 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 1 -- 172.21.15.50:0/12129828 wait complete. 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.282+0000 7ff7c1e2db80 20 client.25618 trim_cache size 0 max 16384 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 25, 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 4, 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "b63a33a6": { 2024-05-10T21:10:23.368 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 4, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 81.3, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:10:23.369 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 msgr2=0x7ff7900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 0x7ff7900b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e2d0 tx=0x7ff7a400d040 comp rx=0 tx=0).stop 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 msgr2=0x7ff7ac10b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 0x7ff7ac10b240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00ed00 comp rx=0 tx=0).stop 2024-05-10T21:10:23.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 shutdown_connections 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7900b2900 0x7ff7900b4e10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac104630 0x7ff7ac10b240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7ac109860 0x7ff7ac10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 --2- 172.21.15.50:0/2123520316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ac108f10 0x7ff7ac10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.285+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 >> 172.21.15.50:0/2123520316 conn(0x7ff7ac1000a0 msgr2=0x7ff7ac110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.286+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 shutdown_connections 2024-05-10T21:10:23.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:23.286+0000 7ff7b3200700 1 -- 172.21.15.50:0/2123520316 wait complete. 2024-05-10T21:10:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:22 smithi142 ceph-mon[34846]: pgmap v505: 97 pgs: 97 active+clean; 673 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T21:10:24.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.341+0000 7fecf7000700 1 -- 172.21.15.50:0/2964358067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105310 msgr2=0x7fecf0105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.341+0000 7fecf7000700 1 --2- 172.21.15.50:0/2964358067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105310 0x7fecf0105720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009990 tx=0x7fecdc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 -- 172.21.15.50:0/2964358067 shutdown_connections 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 --2- 172.21.15.50:0/2964358067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105310 0x7fecf0105720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 --2- 172.21.15.50:0/2964358067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 --2- 172.21.15.50:0/2964358067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105c60 0x7fecf010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 -- 172.21.15.50:0/2964358067 >> 172.21.15.50:0/2964358067 conn(0x7fecf0078ac0 msgr2=0x7fecf0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.342+0000 7fecf7000700 1 -- 172.21.15.50:0/2964358067 shutdown_connections 2024-05-10T21:10:24.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.344+0000 7fecf7000700 1 -- 172.21.15.50:0/2964358067 wait complete. 2024-05-10T21:10:24.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.345+0000 7fecf7000700 1 Processor -- start 2024-05-10T21:10:24.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.345+0000 7fecf7000700 1 -- start start 2024-05-10T21:10:24.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.345+0000 7fecf7000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 0x7fecf0103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.345+0000 7fecf7000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.345+0000 7fecf7000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf7000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01142a0 con 0x7fecf0105310 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf7000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0114120 con 0x7fecf0105c60 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf7000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0114420 con 0x7fecf010e740 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf5200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf5200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40838/0 (socket says 172.21.15.50:40838) 2024-05-10T21:10:24.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf5200700 1 -- 172.21.15.50:0/632561949 learned_addr learned my addr 172.21.15.50:0/632561949 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:24.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf6600700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.346+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 0x7fecf0103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5c00700 1 -- 172.21.15.50:0/632561949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 msgr2=0x7fecf0103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5c00700 1 -- 172.21.15.50:0/632561949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 msgr2=0x7fecf0103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5c00700 1 -- 172.21.15.50:0/632561949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf01a9030 con 0x7fecf0105310 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf6600700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.347+0000 7fecf5200700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:24.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.348+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 0x7fecf0103000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0098e0 tx=0x7fecdc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.348+0000 7fecee000700 1 -- 172.21.15.50:0/632561949 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc027070 con 0x7fecf0105310 2024-05-10T21:10:24.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.348+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf01a92c0 con 0x7fecf0105310 2024-05-10T21:10:24.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.348+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01a97f0 con 0x7fecf0105310 2024-05-10T21:10:24.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.349+0000 7fecee000700 1 -- 172.21.15.50:0/632561949 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecdc022470 con 0x7fecf0105310 2024-05-10T21:10:24.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.349+0000 7fecee000700 1 -- 172.21.15.50:0/632561949 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc017650 con 0x7fecf0105310 2024-05-10T21:10:24.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.354+0000 7fed07235b80 1 Processor -- start 2024-05-10T21:10:24.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.354+0000 7fed07235b80 1 -- start start 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccf3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccf4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d98cc0da20 con 0x55d98cb6de40 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d98cbab760 con 0x55d98ccfa1e0 2024-05-10T21:10:24.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fed07235b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d98cc099e0 con 0x55d98ccf4450 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fecee000700 1 -- 172.21.15.50:0/632561949 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fecdc02a070 con 0x7fecf0105310 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49818/0 (socket says 172.21.15.50:49818) 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.355+0000 7fecf6600700 1 -- 172.21.15.50:0/503840451 learned_addr learned my addr 172.21.15.50:0/503840451 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.356+0000 7fecf5200700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccf4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.356+0000 7fecf6600700 1 -- 172.21.15.50:0/503840451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 msgr2=0x55d98ccf4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.356+0000 7fecf6600700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccf4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.356+0000 7fecf6600700 1 -- 172.21.15.50:0/503840451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 msgr2=0x55d98ccf3f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:10:24.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.357+0000 7fecf5c00700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccf3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.357+0000 7fecf6600700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccf3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.357+0000 7fecf6600700 1 -- 172.21.15.50:0/503840451 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d98cb5a8d0 con 0x55d98cb6de40 2024-05-10T21:10:24.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.357+0000 7fecf5200700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccf4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.357+0000 7fecf6600700 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fece8009550 tx=0x7fece80079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80461628f19b7053 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.358+0000 7fece2a00700 1 -- 172.21.15.50:0/503840451 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8019030 con 0x55d98cb6de40 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.358+0000 7fece2a00700 1 -- 172.21.15.50:0/503840451 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece8015650 con 0x55d98cb6de40 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.358+0000 7fece2a00700 1 -- 172.21.15.50:0/503840451 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8004280 con 0x55d98cb6de40 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.358+0000 7fecee000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 0x7fecd80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fecee000700 1 -- 172.21.15.50:0/632561949 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fecdc0e2fc0 con 0x7fecf0105310 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 -- 172.21.15.50:0/503840451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 msgr2=0x55d98ccf9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fece8009550 tx=0x7fece80079f0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 -- 172.21.15.50:0/503840451 shutdown_connections 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccf4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccf9ca0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.359+0000 7fed07235b80 1 --2- 172.21.15.50:0/503840451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccf3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.360+0000 7fed07235b80 1 -- 172.21.15.50:0/503840451 >> 172.21.15.50:0/503840451 conn(0x55d98cbb2f70 msgr2=0x55d98cce6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.363+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 0x7fecd80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.363+0000 7fed07235b80 1 -- 172.21.15.50:0/503840451 shutdown_connections 2024-05-10T21:10:24.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.364+0000 7fed07235b80 1 -- 172.21.15.50:0/503840451 wait complete. 2024-05-10T21:10:24.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.364+0000 7fed07235b80 1 -- start start 2024-05-10T21:10:24.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.365+0000 7fed07235b80 1 Processor -- start 2024-05-10T21:10:24.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.365+0000 7fecf5c00700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 0x7fecd80b4e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0aa5d0 tx=0x7fecdc005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.365+0000 7fed07235b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccfdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d98cd00ce0 con 0x55d98cb6de40 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d98cb73b20 con 0x55d98ccfa1e0 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d98cd00e60 con 0x55d98ccf4450 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fed07235b80 10 client.0 _ll_register_callbacks cb 0x55d98cbb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49828/0 (socket says 172.21.15.50:49828) 2024-05-10T21:10:24.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.366+0000 7fecf6600700 1 -- 172.21.15.50:0/2149818635 learned_addr learned my addr 172.21.15.50:0/2149818635 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf5200700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccfdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 -- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 msgr2=0x55d98ccfdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccfdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 -- 172.21.15.50:0/2149818635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 msgr2=0x55d98ccfe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.367+0000 7fecf6600700 1 -- 172.21.15.50:0/2149818635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d98cbd3420 con 0x55d98cb6de40 2024-05-10T21:10:24.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.368+0000 7fecf6600700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fece8015f10 tx=0x7fece8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847687eaf1d2de59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.368+0000 7fecd7400700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:24.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.368+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8003c00 con 0x55d98cb6de40 2024-05-10T21:10:24.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.368+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fece8003da0 con 0x55d98cb6de40 2024-05-10T21:10:24.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.369+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece8015650 con 0x55d98cb6de40 2024-05-10T21:10:24.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.372+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fece8031050 con 0x55d98cb6de40 2024-05-10T21:10:24.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.373+0000 7fed07235b80 10 client.15519 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:24.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.373+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d98cd0b8d0 con 0x55d98cb6de40 2024-05-10T21:10:24.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece8015650 con 0x55d98cb6de40 2024-05-10T21:10:24.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fece802a3e0 con 0x55d98cb6de40 2024-05-10T21:10:24.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 10 client.15519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 20 client.15519 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd0dda0 0x55d98cd101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 4 client.15519 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.374+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d98ccb81c0 con 0x55d98cd0dda0 2024-05-10T21:10:24.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.375+0000 7fecf5200700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd0dda0 0x55d98cd101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.376+0000 7fecf5200700 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd0dda0 0x55d98cd101c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4c8951c064bc86f server_cookie=275dacb25073ad21 in_seq=0 out_seq=0 2024-05-10T21:10:24.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.376+0000 7fecd7400700 10 client.15519 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:24.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.376+0000 7fecd7400700 1 -- 172.21.15.50:0/2149818635 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d98ccb81c0 con 0x55d98cd0dda0 2024-05-10T21:10:24.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.376+0000 7fecd7400700 10 client.15519 handle_command_reply: tid=0 2024-05-10T21:10:24.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 client.15519 shutdown 2024-05-10T21:10:24.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 msgr2=0x55d98ccfda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fece8015f10 tx=0x7fece8009d40 comp rx=0 tx=0).stop 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 shutdown_connections 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd0dda0 0x55d98cd101c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fecd7400700 0 client.15519 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98ccfda50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 --2- 172.21.15.50:0/2149818635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98ccfdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.377+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 >> 172.21.15.50:0/2149818635 conn(0x55d98cbb2f70 msgr2=0x55d98ccea6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.378+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 shutdown_connections 2024-05-10T21:10:24.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.378+0000 7fed07235b80 1 -- 172.21.15.50:0/2149818635 wait complete. 2024-05-10T21:10:24.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.378+0000 7fed07235b80 20 client.15519 trim_cache size 0 max 16384 2024-05-10T21:10:24.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.396+0000 7fed07235b80 1 Processor -- start 2024-05-10T21:10:24.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.396+0000 7fed07235b80 1 -- start start 2024-05-10T21:10:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98cb6de40 0x55d98ccfcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000fb30 con 0x55d98ccf4450 2024-05-10T21:10:24.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000d3c0 con 0x55d98ccfa1e0 2024-05-10T21:10:24.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fed07235b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000b210 con 0x55d98cb6de40 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fecf5200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fecf5200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49840/0 (socket says 172.21.15.50:49840) 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.397+0000 7fecf5200700 1 -- 172.21.15.50:0/3825929019 learned_addr learned my addr 172.21.15.50:0/3825929019 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf6600700 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98cb6de40 0x55d98ccfcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 -- 172.21.15.50:0/3825929019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98cb6de40 msgr2=0x55d98ccfcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98cb6de40 0x55d98ccfcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 -- 172.21.15.50:0/3825929019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 msgr2=0x55d98ccfb600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfb600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 -- 172.21.15.50:0/3825929019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d98cce2c40 con 0x55d98ccf4450 2024-05-10T21:10:24.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.398+0000 7fecf5200700 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118f0626c37e0ff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.399+0000 7fece2000700 1 -- 172.21.15.50:0/3825929019 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4013d60 con 0x55d98ccf4450 2024-05-10T21:10:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.399+0000 7fece2000700 1 -- 172.21.15.50:0/3825929019 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece401fe50 con 0x55d98ccf4450 2024-05-10T21:10:24.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.399+0000 7fece2000700 1 -- 172.21.15.50:0/3825929019 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece401f9e0 con 0x55d98ccf4450 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.400+0000 7fed07235b80 1 -- 172.21.15.50:0/3825929019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 msgr2=0x55d98cce0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.400+0000 7fed07235b80 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4009fd0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 -- 172.21.15.50:0/3825929019 shutdown_connections 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98ccfb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98cb6de40 0x55d98ccfcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 --2- 172.21.15.50:0/3825929019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98ccf4450 0x55d98cce0cd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 -- 172.21.15.50:0/3825929019 >> 172.21.15.50:0/3825929019 conn(0x55d98cbb2f70 msgr2=0x55d98ccf7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 -- 172.21.15.50:0/3825929019 shutdown_connections 2024-05-10T21:10:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.401+0000 7fed07235b80 1 -- 172.21.15.50:0/3825929019 wait complete. 2024-05-10T21:10:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.402+0000 7fed07235b80 1 -- start start 2024-05-10T21:10:24.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.402+0000 7fed07235b80 1 Processor -- start 2024-05-10T21:10:24.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98cd06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98cd14c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98cd151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000b880 con 0x55d98cb6de40 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000ccf0 con 0x55d98ccfa1e0 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fecd000bf50 con 0x55d98ccf4450 2024-05-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.403+0000 7fed07235b80 10 client.0 _ll_register_callbacks cb 0x55d98cbb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.404+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98cd06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.404+0000 7fecf6600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98cd06550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49848/0 (socket says 172.21.15.50:49848) 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.404+0000 7fecf6600700 1 -- 172.21.15.50:0/15327130 learned_addr learned my addr 172.21.15.50:0/15327130 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.404+0000 7fecf6600700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98cd151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.404+0000 7fecf5200700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98cd14c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 -- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 msgr2=0x55d98cd14c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98cd14c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 -- 172.21.15.50:0/15327130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 msgr2=0x55d98cd06550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98cd06550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 -- 172.21.15.50:0/15327130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d98cd15780 con 0x55d98ccfa1e0 2024-05-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecf6600700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98cd151c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fece8009190 tx=0x7fece8009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee592a28065c6e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecd7400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8019040 con 0x55d98ccfa1e0 2024-05-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.405+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fece8007df0 con 0x55d98ccfa1e0 2024-05-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.406+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece401fe50 con 0x55d98ccfa1e0 2024-05-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.406+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fece8005e60 con 0x55d98ccfa1e0 2024-05-10T21:10:24.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.406+0000 7fed07235b80 10 client.25649 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:24.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.406+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d98cd78190 con 0x55d98ccfa1e0 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.407+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece401fe50 con 0x55d98ccfa1e0 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.407+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fece802a9a0 con 0x55d98ccfa1e0 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 10 client.25649 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 10 client.25649 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 20 client.25649 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd78840 0x55d98cd7ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 4 client.25649 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d98cc1b400 con 0x55d98cd78840 2024-05-10T21:10:24.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.408+0000 7fecf5200700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd78840 0x55d98cd7ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:24.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.409+0000 7fecd7400700 10 client.25649 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:24.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.409+0000 7fecf5200700 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd78840 0x55d98cd7ac60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ddfb8465df9213d server_cookie=80236652a79a1a54 in_seq=0 out_seq=0 2024-05-10T21:10:24.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.409+0000 7fecd7400700 1 -- 172.21.15.50:0/15327130 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d98cc1b400 con 0x55d98cd78840 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.409+0000 7fecd7400700 10 client.25649 handle_command_reply: tid=0 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.409+0000 7fed07235b80 1 client.25649 shutdown 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 msgr2=0x55d98cd151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98cd151c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fece8009190 tx=0x7fece8009d40 comp rx=0 tx=0).stop 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 shutdown_connections 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d98cd78840 0x55d98cd7ac60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d98ccfa1e0 0x55d98cd151c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fecd7400700 0 client.25649 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d98cb6de40 0x55d98cd06550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 --2- 172.21.15.50:0/15327130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d98ccf4450 0x55d98cd14c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.410+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 >> 172.21.15.50:0/15327130 conn(0x55d98cbb2f70 msgr2=0x55d98cce0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.411+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 shutdown_connections 2024-05-10T21:10:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.411+0000 7fed07235b80 1 -- 172.21.15.50:0/15327130 wait complete. 2024-05-10T21:10:24.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.411+0000 7fed07235b80 20 client.25649 trim_cache size 0 max 16384 2024-05-10T21:10:24.411 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 msgr2=0x7fecd80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 0x7fecd80b4e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0aa5d0 tx=0x7fecdc005e80 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 msgr2=0x7fecf0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 0x7fecf0103000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0098e0 tx=0x7fecdc0057c0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 shutdown_connections 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fecd80b2980 0x7fecd80b4e90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecf010e740 0x7fecf0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecf0105310 0x7fecf0103000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 --2- 172.21.15.50:0/632561949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf0105c60 0x7fecf0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:24.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 >> 172.21.15.50:0/632561949 conn(0x7fecf0078ac0 msgr2=0x7fecf00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:24.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 shutdown_connections 2024-05-10T21:10:24.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:24.412+0000 7fecf7000700 1 -- 172.21.15.50:0/632561949 wait complete. 2024-05-10T21:10:24.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b63a33a6', seconds elapsed: 45.2 2024-05-10T21:10:24.782 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.35 2024-05-10T21:10:24.782 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:10:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:24 smithi050 ceph-mon[29546]: pgmap v506: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:25.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:24 smithi092 ceph-mon[34415]: pgmap v506: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:24 smithi142 ceph-mon[34846]: pgmap v506: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:25.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:10:25.779 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:26.656 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:27.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:26 smithi050 ceph-mon[29546]: pgmap v507: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:27.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:26 smithi092 ceph-mon[34415]: pgmap v507: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:26 smithi142 ceph-mon[34846]: pgmap v507: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3612636115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00109680 msgr2=0x7fea00110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3612636115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00109680 0x7fea00110fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800b0a0 tx=0x7fe9f800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3612636115 shutdown_connections 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3612636115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00109680 0x7fea00110fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3612636115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00104400 0x7fea00104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3612636115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00108d30 0x7fea00109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.414+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3612636115 >> 172.21.15.50:0/3612636115 conn(0x7fea00079d10 msgr2=0x7fea0007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.415+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3612636115 shutdown_connections 2024-05-10T21:10:28.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.415+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3612636115 wait complete. 2024-05-10T21:10:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.417+0000 7fe9ffe00700 1 Processor -- start 2024-05-10T21:10:28.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.417+0000 7fe9ffe00700 1 -- start start 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.417+0000 7fe9ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 0x7fea00077610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9ffe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 0x7fea00075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9ffe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 0x7fea00076060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9ffe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114300 con 0x7fea00109680 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9ffe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114180 con 0x7fea00108d30 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9ffe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114480 con 0x7fea00104400 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9fea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 0x7fea00077610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9fea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 0x7fea00077610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48292/0 (socket says 172.21.15.50:48292) 2024-05-10T21:10:28.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9fea00700 1 -- 172.21.15.50:0/3442728827 learned_addr learned my addr 172.21.15.50:0/3442728827 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.418+0000 7fe9fe000700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 0x7fea00075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:28.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9ff400700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 0x7fea00076060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9fe000700 1 -- 172.21.15.50:0/3442728827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 msgr2=0x7fea00077610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9fe000700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 0x7fea00077610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9fe000700 1 -- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 msgr2=0x7fea00076060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9fe000700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 0x7fea00076060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9fe000700 1 -- 172.21.15.50:0/3442728827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea00076890 con 0x7fea00108d30 2024-05-10T21:10:28.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.419+0000 7fe9ff400700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 0x7fea00076060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:28.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.420+0000 7fe9fe000700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 0x7fea00075ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9c0 tx=0x7fe9ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:28.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.420+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00cbd0 con 0x7fea00108d30 2024-05-10T21:10:28.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.420+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ec00cd70 con 0x7fea00108d30 2024-05-10T21:10:28.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.421+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec018680 con 0x7fea00108d30 2024-05-10T21:10:28.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.421+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0019eec0 con 0x7fea00108d30 2024-05-10T21:10:28.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.421+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0019f450 con 0x7fea00108d30 2024-05-10T21:10:28.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.423+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea001058b0 con 0x7fea00108d30 2024-05-10T21:10:28.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.427+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe9ec04aab0 con 0x7fea00108d30 2024-05-10T21:10:28.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.430+0000 7fe9f7400700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 0x7fe9e40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:28.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.430+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fe9ec0e22c0 con 0x7fea00108d30 2024-05-10T21:10:28.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.431+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9ec0a9fb0 con 0x7fea00108d30 2024-05-10T21:10:28.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.431+0000 7fe9fea00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 0x7fe9e40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:28.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.439+0000 7fe9fea00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 0x7fe9e40b4e90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009d20 tx=0x7fe9f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:28.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.984+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea0002e5f0 con 0x7fea00108d30 2024-05-10T21:10:28.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.986+0000 7fe9f7400700 1 -- 172.21.15.50:0/3442728827 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fe9ec0a98b0 con 0x7fea00108d30 2024-05-10T21:10:28.985 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:10:28.986 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:10:28.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.989+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 msgr2=0x7fe9e40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.989+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 0x7fe9e40b4e90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009d20 tx=0x7fe9f0009260 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.989+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 msgr2=0x7fea00075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.989+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 0x7fea00075ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9c0 tx=0x7fe9ec00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 shutdown_connections 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9e40b2980 0x7fe9e40b4e90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea00109680 0x7fea00076060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea00104400 0x7fea00077610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 --2- 172.21.15.50:0/3442728827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea00108d30 0x7fea00075ac0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9c0 tx=0x7fe9ec00ecd0 comp rx=0 tx=0).stop 2024-05-10T21:10:28.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 >> 172.21.15.50:0/3442728827 conn(0x7fea00079d10 msgr2=0x7fea00101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:28.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.990+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 shutdown_connections 2024-05-10T21:10:28.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:28.991+0000 7fe9ffe00700 1 -- 172.21.15.50:0/3442728827 wait complete. 2024-05-10T21:10:28.991 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:10:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:28 smithi050 ceph-mon[29546]: pgmap v508: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:29.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:28 smithi092 ceph-mon[34415]: pgmap v508: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:28 smithi142 ceph-mon[34846]: pgmap v508: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:29.810 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 60 --expiration 81.35 --await-for 42.0' 2024-05-10T21:10:29.810 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db / --timeout 60 --expiration 81.35 --await-for 42.0 2024-05-10T21:10:30.126 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:29 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3442728827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:29 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3442728827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:29 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3442728827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:30.451 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 -- 172.21.15.50:0/261031215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9654103080 msgr2=0x7f9654103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 --2- 172.21.15.50:0/261031215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9654103080 0x7f9654103490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96440098f0 tx=0x7f9644009c00 comp rx=0 tx=0).stop 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 -- 172.21.15.50:0/261031215 shutdown_connections 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 --2- 172.21.15.50:0/261031215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9654103080 0x7f9654103490 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 --2- 172.21.15.50:0/261031215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96541084d0 0x7f96541089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 --2- 172.21.15.50:0/261031215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541039d0 0x7f9654107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.504+0000 7f9653200700 1 -- 172.21.15.50:0/261031215 >> 172.21.15.50:0/261031215 conn(0x7f9654078ac0 msgr2=0x7f9654078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.505+0000 7f9653200700 1 -- 172.21.15.50:0/261031215 shutdown_connections 2024-05-10T21:10:30.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.505+0000 7f9653200700 1 -- 172.21.15.50:0/261031215 wait complete. 2024-05-10T21:10:30.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.506+0000 7f9653200700 1 Processor -- start 2024-05-10T21:10:30.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 -- start start 2024-05-10T21:10:30.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96541039d0 0x7f96541a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541084d0 0x7f96541a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f965411a8b0 con 0x7f9654103080 2024-05-10T21:10:30.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f965411a730 con 0x7f96541039d0 2024-05-10T21:10:30.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.507+0000 7f9653200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f965411aa30 con 0x7f96541084d0 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.508+0000 7f9651e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.508+0000 7f9651e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49890/0 (socket says 172.21.15.50:49890) 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.508+0000 7f9651e00700 1 -- 172.21.15.50:0/3773091774 learned_addr learned my addr 172.21.15.50:0/3773091774 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9652800700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541084d0 0x7f96541a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651400700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96541039d0 0x7f96541a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651e00700 1 -- 172.21.15.50:0/3773091774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541084d0 msgr2=0x7f96541a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651e00700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541084d0 0x7f96541a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651e00700 1 -- 172.21.15.50:0/3773091774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96541039d0 msgr2=0x7f96541a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651e00700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96541039d0 0x7f96541a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.509+0000 7f9651e00700 1 -- 172.21.15.50:0/3773091774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541a6240 con 0x7f9654103080 2024-05-10T21:10:30.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.510+0000 7f9651e00700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f9644005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.510+0000 7f964aa00700 1 -- 172.21.15.50:0/3773091774 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644027070 con 0x7f9654103080 2024-05-10T21:10:30.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.511+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96541a64d0 con 0x7f9654103080 2024-05-10T21:10:30.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.511+0000 7f964aa00700 1 -- 172.21.15.50:0/3773091774 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964400b560 con 0x7f9654103080 2024-05-10T21:10:30.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.511+0000 7f964aa00700 1 -- 172.21.15.50:0/3773091774 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644004a50 con 0x7f9654103080 2024-05-10T21:10:30.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.511+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541ade50 con 0x7f9654103080 2024-05-10T21:10:30.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.520+0000 7f964aa00700 1 -- 172.21.15.50:0/3773091774 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f964402a070 con 0x7f9654103080 2024-05-10T21:10:30.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.520+0000 7f9667d78b80 1 Processor -- start 2024-05-10T21:10:30.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 -- start start 2024-05-10T21:10:30.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91ddb7e40 0x55f91df35d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 0x55f91df3f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 0x55f91df41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f91de57a20 con 0x55f91df36690 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f91ddf5760 con 0x55f91df36280 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.521+0000 7f9667d78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f91de539e0 con 0x55f91ddb7e40 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.522+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 0x55f91df3f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.522+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 0x55f91df3f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40920/0 (socket says 172.21.15.50:40920) 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.522+0000 7f9651e00700 1 -- 172.21.15.50:0/95839642 learned_addr learned my addr 172.21.15.50:0/95839642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.522+0000 7f9652800700 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91ddb7e40 0x55f91df35d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.523+0000 7f9651400700 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 0x55f91df41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 -- 172.21.15.50:0/95839642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91ddb7e40 msgr2=0x55f91df35d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91ddb7e40 0x55f91df35d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 -- 172.21.15.50:0/95839642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 msgr2=0x55f91df3f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 0x55f91df3f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 -- 172.21.15.50:0/95839642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f91dda48d0 con 0x55f91df36690 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f964aa00700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 0x7f96380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.524+0000 7f9651400700 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 0x55f91df41e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9640005fd0 tx=0x7f96400113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f23b8090780d350 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.525+0000 7f964aa00700 1 -- 172.21.15.50:0/3773091774 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f96440e2c80 con 0x7f9654103080 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.525+0000 7f963f400700 1 -- 172.21.15.50:0/95839642 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640011c60 con 0x55f91df36690 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.525+0000 7f963f400700 1 -- 172.21.15.50:0/95839642 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640011e00 con 0x55f91df36690 2024-05-10T21:10:30.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.525+0000 7f963f400700 1 -- 172.21.15.50:0/95839642 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640012450 con 0x55f91df36690 2024-05-10T21:10:30.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.526+0000 7f9652800700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 0x7f96380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.526+0000 7f9667d78b80 1 -- 172.21.15.50:0/95839642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 msgr2=0x55f91df41e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.526+0000 7f9667d78b80 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 0x55f91df41e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9640005fd0 tx=0x7f96400113e0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.527+0000 7f9652800700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 0x7f96380b4e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f91df31500 tx=0x7f964c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.527+0000 7f9667d78b80 1 -- 172.21.15.50:0/95839642 shutdown_connections 2024-05-10T21:10:30.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.527+0000 7f9667d78b80 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36690 0x55f91df41e30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.527+0000 7f9667d78b80 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91ddb7e40 0x55f91df35d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.527+0000 7f9667d78b80 1 --2- 172.21.15.50:0/95839642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91df36280 0x55f91df3f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.528+0000 7f9667d78b80 1 -- 172.21.15.50:0/95839642 >> 172.21.15.50:0/95839642 conn(0x55f91ddf6a80 msgr2=0x55f91df30b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.528+0000 7f9667d78b80 1 -- 172.21.15.50:0/95839642 shutdown_connections 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.528+0000 7f9667d78b80 1 -- 172.21.15.50:0/95839642 wait complete. 2024-05-10T21:10:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.529+0000 7f9667d78b80 1 -- start start 2024-05-10T21:10:30.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.529+0000 7f9667d78b80 1 Processor -- start 2024-05-10T21:10:30.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f91df427a0 con 0x55f91df36280 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f91ddbdb20 con 0x55f91ddb7e40 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f91df42920 con 0x55f91df36690 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9667d78b80 10 client.0 _ll_register_callbacks cb 0x55f91ddeb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9651400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9651400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49896/0 (socket says 172.21.15.50:49896) 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.530+0000 7f9651400700 1 -- 172.21.15.50:0/4282065491 learned_addr learned my addr 172.21.15.50:0/4282065491 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.531+0000 7f9651400700 1 -- 172.21.15.50:0/4282065491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 msgr2=0x55f91df3fdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.531+0000 7f9651400700 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.531+0000 7f9651400700 1 -- 172.21.15.50:0/4282065491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 msgr2=0x55f91df3f330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.531+0000 7f9651400700 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.531+0000 7f9651400700 1 -- 172.21.15.50:0/4282065491 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f91de1d420 con 0x55f91df36280 2024-05-10T21:10:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.532+0000 7f9651400700 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9640009630 tx=0x7f9640011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2a736009534c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.532+0000 7f9633e00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.532+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640004020 con 0x55f91df36280 2024-05-10T21:10:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.533+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f964001f070 con 0x55f91df36280 2024-05-10T21:10:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.533+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9640011e00 con 0x55f91df36280 2024-05-10T21:10:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.533+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96400041c0 con 0x55f91df36280 2024-05-10T21:10:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.533+0000 7f9667d78b80 10 client.15543 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.533+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f91df4d3d0 con 0x55f91df36280 2024-05-10T21:10:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.534+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640011e00 con 0x55f91df36280 2024-05-10T21:10:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9640014620 con 0x55f91df36280 2024-05-10T21:10:30.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:30.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 10 client.15543 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:10:30.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 20 client.15543 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:30.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91df4f8f0 0x55f91df51d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 4 client.15543 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:30.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.535+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f91df02070 con 0x55f91df4f8f0 2024-05-10T21:10:30.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.536+0000 7f9651e00700 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91df4f8f0 0x55f91df51d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.537+0000 7f9651e00700 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91df4f8f0 0x55f91df51d10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b796094ed837e16 server_cookie=1bbf41a2e3e20f2 in_seq=0 out_seq=0 2024-05-10T21:10:30.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.537+0000 7f9633e00700 10 client.15543 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.539+0000 7f9633e00700 1 -- 172.21.15.50:0/4282065491 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f91df02070 con 0x55f91df4f8f0 2024-05-10T21:10:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.539+0000 7f9633e00700 10 client.15543 handle_command_reply: tid=0 2024-05-10T21:10:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.539+0000 7f9667d78b80 1 client.15543 shutdown 2024-05-10T21:10:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 msgr2=0x55f91df3f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9640009630 tx=0x7f9640011cf0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 shutdown_connections 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91df4f8f0 0x55f91df51d10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 --2- 172.21.15.50:0/4282065491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df3f870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.540+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 >> 172.21.15.50:0/4282065491 conn(0x55f91ddf6a80 msgr2=0x55f91df34700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.541+0000 7f9633e00700 0 client.15543 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:30.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.543+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 shutdown_connections 2024-05-10T21:10:30.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.544+0000 7f9667d78b80 1 -- 172.21.15.50:0/4282065491 wait complete. 2024-05-10T21:10:30.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.544+0000 7f9667d78b80 20 client.15543 trim_cache size 0 max 16384 2024-05-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.560+0000 7f9667d78b80 1 Processor -- start 2024-05-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.560+0000 7f9667d78b80 1 -- start start 2024-05-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96541bffb0 con 0x55f91df36280 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9654074200 con 0x55f91ddb7e40 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9667d78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96540720b0 con 0x55f91df36690 2024-05-10T21:10:30.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9652800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9652800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48346/0 (socket says 172.21.15.50:48346) 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9652800700 1 -- 172.21.15.50:0/2072035817 learned_addr learned my addr 172.21.15.50:0/2072035817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40938/0 (socket says 172.21.15.50:40938) 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.561+0000 7f9651400700 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9652800700 1 -- 172.21.15.50:0/2072035817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 msgr2=0x55f91df3fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9652800700 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9652800700 1 -- 172.21.15.50:0/2072035817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 msgr2=0x55f91df410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9652800700 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9652800700 1 -- 172.21.15.50:0/2072035817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f91df3e650 con 0x55f91df36690 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.562+0000 7f9651400700 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.563+0000 7f9652800700 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964c007fd0 tx=0x7f964c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2689bf3120a801fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.563+0000 7f963ea00700 1 -- 172.21.15.50:0/2072035817 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c01e050 con 0x55f91df36690 2024-05-10T21:10:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.563+0000 7f963ea00700 1 -- 172.21.15.50:0/2072035817 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c028070 con 0x55f91df36690 2024-05-10T21:10:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.564+0000 7f963ea00700 1 -- 172.21.15.50:0/2072035817 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c016680 con 0x55f91df36690 2024-05-10T21:10:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 -- 172.21.15.50:0/2072035817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 msgr2=0x55f91df3a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964c007fd0 tx=0x7f964c015000 comp rx=0 tx=0).stop 2024-05-10T21:10:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 -- 172.21.15.50:0/2072035817 shutdown_connections 2024-05-10T21:10:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df3fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91df3a6c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 --2- 172.21.15.50:0/2072035817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 -- 172.21.15.50:0/2072035817 >> 172.21.15.50:0/2072035817 conn(0x55f91ddf6a80 msgr2=0x55f91df409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.565+0000 7f9667d78b80 1 -- 172.21.15.50:0/2072035817 shutdown_connections 2024-05-10T21:10:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.566+0000 7f9667d78b80 1 -- 172.21.15.50:0/2072035817 wait complete. 2024-05-10T21:10:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.566+0000 7f9667d78b80 1 -- start start 2024-05-10T21:10:30.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.567+0000 7f9667d78b80 1 Processor -- start 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.567+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.567+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9667d78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91dfb7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9667d78b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9654072700 con 0x55f91df36280 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9667d78b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9654073b30 con 0x55f91ddb7e40 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9667d78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9654072d90 con 0x55f91df36690 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9667d78b80 10 client.0 _ll_register_callbacks cb 0x55f91ddeb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9651e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40954/0 (socket says 172.21.15.50:40954) 2024-05-10T21:10:30.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9651e00700 1 -- 172.21.15.50:0/3945180735 learned_addr learned my addr 172.21.15.50:0/3945180735 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:30.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9651400700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9652800700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91dfb7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.568+0000 7f9651e00700 1 -- 172.21.15.50:0/3945180735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 msgr2=0x55f91dfb7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651e00700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91dfb7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651e00700 1 -- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 msgr2=0x55f91df52240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651e00700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df52240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651e00700 1 -- 172.21.15.50:0/3945180735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f91dfb83c0 con 0x55f91ddb7e40 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651400700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df52240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9652800700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91dfb7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9651e00700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96440aa2f0 tx=0x7f96440e6760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f256cb70f7356c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.569+0000 7f9633e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:30.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440a9ad0 con 0x55f91ddb7e40 2024-05-10T21:10:30.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9644004bf0 con 0x55f91ddb7e40 2024-05-10T21:10:30.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f964c028070 con 0x55f91ddb7e40 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96440a9e70 con 0x55f91ddb7e40 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9667d78b80 10 client.25670 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.570+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f91dfb9680 con 0x55f91ddb7e40 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c028070 con 0x55f91ddb7e40 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f96440aa5c0 con 0x55f91ddb7e40 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 10 client.25670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 20 client.25670 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:30.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91dfb9d80 0x55f91dfbc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:30.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 4 client.25670 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:30.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.571+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f91de65400 con 0x55f91dfb9d80 2024-05-10T21:10:30.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.572+0000 7f9651400700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91dfb9d80 0x55f91dfbc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.573+0000 7f9651400700 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91dfb9d80 0x55f91dfbc1a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e195416295b3920 server_cookie=f094ac919f4c1f78 in_seq=0 out_seq=0 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.573+0000 7f9633e00700 10 client.25670 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.573+0000 7f9633e00700 1 -- 172.21.15.50:0/3945180735 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f91de65400 con 0x55f91dfb9d80 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.573+0000 7f9633e00700 10 client.25670 handle_command_reply: tid=0 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.573+0000 7f9667d78b80 1 client.25670 shutdown 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 msgr2=0x55f91df51d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96440aa2f0 tx=0x7f96440e6760 comp rx=0 tx=0).stop 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 shutdown_connections 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f91ddb7e40 0x55f91df51d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55f91dfb9d80 0x55f91dfbc1a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91df36690 0x55f91dfb7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 --2- 172.21.15.50:0/3945180735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f91df36280 0x55f91df52240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 >> 172.21.15.50:0/3945180735 conn(0x55f91ddf6a80 msgr2=0x55f91df308c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9633e00700 0 client.25670 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:30.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 shutdown_connections 2024-05-10T21:10:30.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.574+0000 7f9667d78b80 1 -- 172.21.15.50:0/3945180735 wait complete. 2024-05-10T21:10:30.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.575+0000 7f9667d78b80 20 client.25670 trim_cache size 0 max 16384 2024-05-10T21:10:30.575 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:30.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.575+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 msgr2=0x7f96380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.575+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 0x7f96380b4e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f91df31500 tx=0x7f964c009040 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.575+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 msgr2=0x7f96541a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.575+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f9644005e40 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 shutdown_connections 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f96380b2900 0x7f96380b4e10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96541084d0 0x7f96541a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9654103080 0x7f96541a13c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 --2- 172.21.15.50:0/3773091774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96541039d0 0x7f96541a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 >> 172.21.15.50:0/3773091774 conn(0x7f9654078ac0 msgr2=0x7f9654105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.576+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 shutdown_connections 2024-05-10T21:10:30.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:30.577+0000 7f9653200700 1 -- 172.21.15.50:0/3773091774 wait complete. 2024-05-10T21:10:30.908 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:30 smithi050 ceph-mon[29546]: pgmap v509: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:31.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:30 smithi092 ceph-mon[34415]: pgmap v509: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:30 smithi142 ceph-mon[34846]: pgmap v509: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:31.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:10:32.707 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:33.147 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:32 smithi050 ceph-mon[29546]: pgmap v510: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:33.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:32 smithi092 ceph-mon[34415]: pgmap v510: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:32 smithi142 ceph-mon[34846]: pgmap v510: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 -- 172.21.15.50:0/467987869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c1019a0 msgr2=0x7f686c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 --2- 172.21.15.50:0/467987869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c1019a0 0x7f686c101db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02a400 comp rx=0 tx=0).stop 2024-05-10T21:10:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 -- 172.21.15.50:0/467987869 shutdown_connections 2024-05-10T21:10:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 --2- 172.21.15.50:0/467987869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c102720 0x7f686c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 --2- 172.21.15.50:0/467987869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 --2- 172.21.15.50:0/467987869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c1019a0 0x7f686c101db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 -- 172.21.15.50:0/467987869 >> 172.21.15.50:0/467987869 conn(0x7f686c0fd080 msgr2=0x7f686c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 -- 172.21.15.50:0/467987869 shutdown_connections 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.797+0000 7f6872800640 1 -- 172.21.15.50:0/467987869 wait complete. 2024-05-10T21:10:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.799+0000 7f6872800640 1 Processor -- start 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.799+0000 7f6872800640 1 -- start start 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c1019a0 0x7f686c108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 0x7f686c108d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1119c0 con 0x7f686c103070 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f686c111840 con 0x7f686c102720 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6872800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f686c111b40 con 0x7f686c1019a0 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6863e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 0x7f686c108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6863e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 0x7f686c108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40980/0 (socket says 172.21.15.50:40980) 2024-05-10T21:10:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f6863e00640 1 -- 172.21.15.50:0/3733752740 learned_addr learned my addr 172.21.15.50:0/3733752740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.800+0000 7f686be00640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686b400640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c1019a0 0x7f686c108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686be00640 1 -- 172.21.15.50:0/3733752740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c1019a0 msgr2=0x7f686c108830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686be00640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c1019a0 0x7f686c108830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686be00640 1 -- 172.21.15.50:0/3733752740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 msgr2=0x7f686c108d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686be00640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 0x7f686c108d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.801+0000 7f686be00640 1 -- 172.21.15.50:0/3733752740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c105ba0 con 0x7f686c103070 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.802+0000 7f686be00640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c105380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6864009fd0 tx=0x7f686400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.802+0000 7f6868c00640 1 -- 172.21.15.50:0/3733752740 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686400ce20 con 0x7f686c103070 2024-05-10T21:10:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.802+0000 7f6868c00640 1 -- 172.21.15.50:0/3733752740 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864005020 con 0x7f686c103070 2024-05-10T21:10:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.802+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c105e90 con 0x7f686c103070 2024-05-10T21:10:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.802+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c19c4b0 con 0x7f686c103070 2024-05-10T21:10:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.803+0000 7f6868c00640 1 -- 172.21.15.50:0/3733752740 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864010690 con 0x7f686c103070 2024-05-10T21:10:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.807+0000 7f6868c00640 1 -- 172.21.15.50:0/3733752740 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6864010830 con 0x7f686c103070 2024-05-10T21:10:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.809+0000 7f6868c00640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 0x7f68500b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.809+0000 7f687984e740 1 Processor -- start 2024-05-10T21:10:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f6868c00640 1 -- 172.21.15.50:0/3733752740 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f68640e2790 con 0x7f686c103070 2024-05-10T21:10:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 -- start start 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 0x55b3e1b59100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 0x55b3e1b59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1abd550 con 0x55b3e17b71b0 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1a62bc0 con 0x55b3e17964d0 2024-05-10T21:10:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.810+0000 7f687984e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1ab9510 con 0x55b3e178e2f0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40992/0 (socket says 172.21.15.50:40992) 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 -- 172.21.15.50:0/3405956859 learned_addr learned my addr 172.21.15.50:0/3405956859 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686b400640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 0x7f68500b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686b400640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 0x55b3e1b59100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f6863e00640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 0x55b3e1b59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 -- 172.21.15.50:0/3405956859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 msgr2=0x55b3e1b59680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 0x55b3e1b59680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 -- 172.21.15.50:0/3405956859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 msgr2=0x55b3e1b59100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 0x55b3e1b59100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686be00640 1 -- 172.21.15.50:0/3405956859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e17cf210 con 0x55b3e17964d0 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f6863e00640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 0x55b3e1b59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.811+0000 7f686b400640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 0x55b3e1b59100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.812+0000 7f686be00640 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68640a9a80 tx=0x7f68640a99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958db86d4059bd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.812+0000 7f6860c00640 1 -- 172.21.15.50:0/3405956859 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640c4420 con 0x55b3e17964d0 2024-05-10T21:10:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.812+0000 7f6860c00640 1 -- 172.21.15.50:0/3405956859 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640a6370 con 0x55b3e17964d0 2024-05-10T21:10:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.812+0000 7f6860c00640 1 -- 172.21.15.50:0/3405956859 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640b2750 con 0x55b3e17964d0 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 -- 172.21.15.50:0/3405956859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 msgr2=0x55b3e1b60580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f68640a9a80 tx=0x7f68640a99f0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f686b400640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 0x7f68500b4e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f685c031440 tx=0x7f685c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 -- 172.21.15.50:0/3405956859 shutdown_connections 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e178e2f0 0x55b3e1b59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17964d0 0x55b3e1b60580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 --2- 172.21.15.50:0/3405956859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17b71b0 0x55b3e1b59100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 -- 172.21.15.50:0/3405956859 >> 172.21.15.50:0/3405956859 conn(0x55b3e17ab300 msgr2=0x55b3e1b54a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.813+0000 7f687984e740 1 -- 172.21.15.50:0/3405956859 shutdown_connections 2024-05-10T21:10:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.814+0000 7f687984e740 1 -- 172.21.15.50:0/3405956859 wait complete. 2024-05-10T21:10:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.815+0000 7f687984e740 1 -- start start 2024-05-10T21:10:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.815+0000 7f687984e740 1 Processor -- start 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 0x55b3e1b62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17b71b0 0x55b3e1b6a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1b60fd0 con 0x55b3e17964d0 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1a76660 con 0x55b3e178e2f0 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3e1b61150 con 0x55b3e17b71b0 2024-05-10T21:10:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f687984e740 10 client.0 _ll_register_callbacks cb 0x55b3e1a40400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f686b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 0x55b3e1b62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f6863e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f6863e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49960/0 (socket says 172.21.15.50:49960) 2024-05-10T21:10:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.816+0000 7f6863e00640 1 -- 172.21.15.50:0/1679838545 learned_addr learned my addr 172.21.15.50:0/1679838545 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f686be00640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17b71b0 0x55b3e1b6a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f6863e00640 1 -- 172.21.15.50:0/1679838545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17b71b0 msgr2=0x55b3e1b6a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f6863e00640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17b71b0 0x55b3e1b6a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f6863e00640 1 -- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 msgr2=0x55b3e1b62bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f6863e00640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 0x55b3e1b62bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f6863e00640 1 -- 172.21.15.50:0/1679838545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e189b620 con 0x55b3e17964d0 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.817+0000 7f686b400640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 0x55b3e1b62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f6863e00640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f685400bb10 tx=0x7f6854004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422ae9cc960c8515 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f6859600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854011640 con 0x55b3e17964d0 2024-05-10T21:10:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f68540117e0 con 0x55b3e17964d0 2024-05-10T21:10:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68640a6370 con 0x55b3e17964d0 2024-05-10T21:10:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.818+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68540119e0 con 0x55b3e17964d0 2024-05-10T21:10:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f687984e740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3e1b6d000 con 0x55b3e17964d0 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640a6370 con 0x55b3e17964d0 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6854010440 con 0x55b3e17964d0 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f687984e740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f687984e740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.819+0000 7f687984e740 20 client.15558 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.820+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e1775d80 0x55b3e1b707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.820+0000 7f687984e740 4 client.15558 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.820+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3e1acb160 con 0x55b3e1775d80 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.820+0000 7f686b400640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e1775d80 0x55b3e1b707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.821+0000 7f686b400640 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e1775d80 0x55b3e1b707c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a1bc42c218ca6a server_cookie=504fcd080d00c92 in_seq=0 out_seq=0 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.821+0000 7f6859600640 10 client.15558 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f6859600640 1 -- 172.21.15.50:0/1679838545 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3e1acb160 con 0x55b3e1775d80 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f6859600640 10 client.15558 handle_command_reply: tid=0 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 client.15558 shutdown 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 msgr2=0x55b3e1b63100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f685400bb10 tx=0x7f6854004500 comp rx=0 tx=0).stop 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 shutdown_connections 2024-05-10T21:10:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e1775d80 0x55b3e1b707c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17b71b0 0x55b3e1b6a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e17964d0 0x55b3e1b63100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 --2- 172.21.15.50:0/1679838545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e178e2f0 0x55b3e1b62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 >> 172.21.15.50:0/1679838545 conn(0x55b3e17ab300 msgr2=0x55b3e1b579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.822+0000 7f6859600640 0 client.15558 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.823+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 shutdown_connections 2024-05-10T21:10:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.823+0000 7f687984e740 1 -- 172.21.15.50:0/1679838545 wait complete. 2024-05-10T21:10:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.823+0000 7f687984e740 20 client.15558 trim_cache size 0 max 16384 2024-05-10T21:10:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.840+0000 7f687984e740 1 Processor -- start 2024-05-10T21:10:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.840+0000 7f687984e740 1 -- start start 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6848010bf0 con 0x55b3e178e2f0 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f684800e4c0 con 0x55b3e17b71b0 2024-05-10T21:10:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f687984e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f684800c310 con 0x55b3e17964d0 2024-05-10T21:10:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f686b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f686b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b5d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49976/0 (socket says 172.21.15.50:49976) 2024-05-10T21:10:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.841+0000 7f686b400640 1 -- 172.21.15.50:0/1639700094 learned_addr learned my addr 172.21.15.50:0/1639700094 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f6863e00640 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 -- 172.21.15.50:0/1639700094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 msgr2=0x55b3e1b5d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b5d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 -- 172.21.15.50:0/1639700094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 msgr2=0x55b3e1b5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b5d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 -- 172.21.15.50:0/1639700094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e1b637c0 con 0x55b3e17b71b0 2024-05-10T21:10:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.842+0000 7f686be00640 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b69f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6864005b70 tx=0x7f68640a96d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dafa20e94b3fc72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.843+0000 7f685be00640 1 -- 172.21.15.50:0/1639700094 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640b7070 con 0x55b3e17b71b0 2024-05-10T21:10:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.843+0000 7f685be00640 1 -- 172.21.15.50:0/1639700094 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640a6370 con 0x55b3e17b71b0 2024-05-10T21:10:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.843+0000 7f685be00640 1 -- 172.21.15.50:0/1639700094 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864010b20 con 0x55b3e17b71b0 2024-05-10T21:10:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.844+0000 7f687984e740 1 -- 172.21.15.50:0/1639700094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 msgr2=0x55b3e1b69f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.844+0000 7f687984e740 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b69f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6864005b70 tx=0x7f68640a96d0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.844+0000 7f687984e740 1 -- 172.21.15.50:0/1639700094 shutdown_connections 2024-05-10T21:10:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b5d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b69f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 --2- 172.21.15.50:0/1639700094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 -- 172.21.15.50:0/1639700094 >> 172.21.15.50:0/1639700094 conn(0x55b3e17ab300 msgr2=0x55b3e1b4f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 -- 172.21.15.50:0/1639700094 shutdown_connections 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 -- 172.21.15.50:0/1639700094 wait complete. 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.845+0000 7f687984e740 1 -- start start 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 Processor -- start 2024-05-10T21:10:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b67150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f684800c980 con 0x55b3e178e2f0 2024-05-10T21:10:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f684800ddf0 con 0x55b3e17b71b0 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.846+0000 7f687984e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f684800d050 con 0x55b3e17964d0 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f687984e740 10 client.0 _ll_register_callbacks cb 0x7f6854004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f686b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f686b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49984/0 (socket says 172.21.15.50:49984) 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f686b400640 1 -- 172.21.15.50:0/2832285318 learned_addr learned my addr 172.21.15.50:0/2832285318 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f6863e00640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b67150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.847+0000 7f686be00640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 -- 172.21.15.50:0/2832285318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 msgr2=0x55b3e1b67150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b67150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 -- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 msgr2=0x55b3e1b67690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b67690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 -- 172.21.15.50:0/2832285318 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e1b67d00 con 0x55b3e178e2f0 2024-05-10T21:10:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f686b400640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27604f5faad293 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.848+0000 7f6859600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.849+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c035040 con 0x55b3e178e2f0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.849+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f685c051710 con 0x55b3e178e2f0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.849+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68640a6370 con 0x55b3e178e2f0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.849+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f685c051910 con 0x55b3e178e2f0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.850+0000 7f687984e740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.850+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3e1bd7cf0 con 0x55b3e178e2f0 2024-05-10T21:10:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.850+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68640a6370 con 0x55b3e178e2f0 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.850+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f685c051ce0 con 0x55b3e178e2f0 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 20 client.15564 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e174b1c0 0x55b3e1bda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 4 client.15564 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3e1ac4fd0 con 0x55b3e174b1c0 2024-05-10T21:10:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.851+0000 7f6863e00640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e174b1c0 0x55b3e1bda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.852+0000 7f6863e00640 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e174b1c0 0x55b3e1bda760 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf47093aa44a51c server_cookie=3848aaacf4589a0a in_seq=0 out_seq=0 2024-05-10T21:10:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.852+0000 7f6859600640 10 client.15564 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.856+0000 7f6859600640 1 -- 172.21.15.50:0/2832285318 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3e1ac4fd0 con 0x55b3e174b1c0 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.857+0000 7f6859600640 10 client.15564 handle_command_reply: tid=0 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.857+0000 7f687984e740 1 client.15564 shutdown 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 msgr2=0x55b3e1b6a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c009260 comp rx=0 tx=0).stop 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 shutdown_connections 2024-05-10T21:10:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b3e174b1c0 0x55b3e1bda760 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3e17964d0 0x55b3e1b67150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3e17b71b0 0x55b3e1b67690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 --2- 172.21.15.50:0/2832285318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3e178e2f0 0x55b3e1b6a5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 >> 172.21.15.50:0/2832285318 conn(0x55b3e17ab300 msgr2=0x55b3e1b62b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.858+0000 7f6859600640 0 client.15564 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.859+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 shutdown_connections 2024-05-10T21:10:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.859+0000 7f687984e740 1 -- 172.21.15.50:0/2832285318 wait complete. 2024-05-10T21:10:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.859+0000 7f687984e740 20 client.15564 trim_cache size 0 max 16384 2024-05-10T21:10:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:10:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:10:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:10:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:10:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 msgr2=0x7f68500b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 0x7f68500b4e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f685c031440 tx=0x7f685c000bc0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 msgr2=0x7f686c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c105380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6864009fd0 tx=0x7f686400ef10 comp rx=0 tx=0).stop 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 shutdown_connections 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f68500b2930 0x7f68500b4e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f686c103070 0x7f686c105380 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f686c102720 0x7f686c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 --2- 172.21.15.50:0/3733752740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f686c1019a0 0x7f686c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 >> 172.21.15.50:0/3733752740 conn(0x7f686c0fd080 msgr2=0x7f686c10b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.860+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 shutdown_connections 2024-05-10T21:10:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:33.861+0000 7f6872800640 1 -- 172.21.15.50:0/3733752740 wait complete. 2024-05-10T21:10:33.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 -- 172.21.15.50:0/930143313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 msgr2=0x7fde0c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 --2- 172.21.15.50:0/930143313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c109330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddf80099c0 tx=0x7fddf8009cd0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 -- 172.21.15.50:0/930143313 shutdown_connections 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 --2- 172.21.15.50:0/930143313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 --2- 172.21.15.50:0/930143313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 0x7fde0c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 --2- 172.21.15.50:0/930143313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c109330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 -- 172.21.15.50:0/930143313 >> 172.21.15.50:0/930143313 conn(0x7fde0c100040 msgr2=0x7fde0c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 -- 172.21.15.50:0/930143313 shutdown_connections 2024-05-10T21:10:33.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.909+0000 7fde13e00700 1 -- 172.21.15.50:0/930143313 wait complete. 2024-05-10T21:10:33.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 Processor -- start 2024-05-10T21:10:33.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 -- start start 2024-05-10T21:10:33.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 0x7fde0c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1144d0 con 0x7fde0c108f20 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c114350 con 0x7fde0c1045f0 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.911+0000 7fde13e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c114650 con 0x7fde0c109870 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.912+0000 7fde11400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.912+0000 7fde11400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43516/0 (socket says 172.21.15.50:43516) 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.912+0000 7fde11400700 1 -- 172.21.15.50:0/2423319615 learned_addr learned my addr 172.21.15.50:0/2423319615 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.912+0000 7fde10a00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.912+0000 7fde11e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 0x7fde0c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 -- 172.21.15.50:0/2423319615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 msgr2=0x7fde0c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 0x7fde0c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 -- 172.21.15.50:0/2423319615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 msgr2=0x7fde0c19d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 -- 172.21.15.50:0/2423319615 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c1a1f90 con 0x7fde0c108f20 2024-05-10T21:10:33.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde11400700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:33.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.913+0000 7fde10a00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c19d660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddf80098e0 tx=0x7fddf80055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.914+0000 7fde0a000700 1 -- 172.21.15.50:0/2423319615 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8026070 con 0x7fde0c108f20 2024-05-10T21:10:33.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.914+0000 7fde0a000700 1 -- 172.21.15.50:0/2423319615 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf800f460 con 0x7fde0c108f20 2024-05-10T21:10:33.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.914+0000 7fde0a000700 1 -- 172.21.15.50:0/2423319615 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8009e60 con 0x7fde0c108f20 2024-05-10T21:10:33.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.914+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c1a2220 con 0x7fde0c108f20 2024-05-10T21:10:33.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.915+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a26b0 con 0x7fde0c108f20 2024-05-10T21:10:33.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.921+0000 7fde0a000700 1 -- 172.21.15.50:0/2423319615 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fddf8029070 con 0x7fde0c108f20 2024-05-10T21:10:33.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.922+0000 7fde22a1ab80 1 Processor -- start 2024-05-10T21:10:33.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.923+0000 7fde0a000700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 0x7fddf40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.923+0000 7fde0a000700 1 -- 172.21.15.50:0/2423319615 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fddf80e2a20 con 0x7fde0c108f20 2024-05-10T21:10:33.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.923+0000 7fde22a1ab80 1 -- start start 2024-05-10T21:10:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde11e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 0x7fddf40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 0x55738f4edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55738f405bb0 con 0x55738f365fd0 2024-05-10T21:10:33.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55738f3a38f0 con 0x55738f4e3aa0 2024-05-10T21:10:33.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55738f401b70 con 0x55738f4e3690 2024-05-10T21:10:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde11400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.924+0000 7fde10a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 0x55738f4edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde11400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50436/0 (socket says 172.21.15.50:50436) 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde11400700 1 -- 172.21.15.50:0/831738278 learned_addr learned my addr 172.21.15.50:0/831738278 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde11e00700 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde10a00700 1 -- 172.21.15.50:0/831738278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 msgr2=0x55738f4f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde10a00700 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde10a00700 1 -- 172.21.15.50:0/831738278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 msgr2=0x55738f4e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde10a00700 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde10a00700 1 -- 172.21.15.50:0/831738278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738f352a60 con 0x55738f4e3690 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.925+0000 7fde11400700 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.926+0000 7fde11e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 0x7fddf40b4e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde04009fd0 tx=0x7fde0400f8e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.926+0000 7fde10a00700 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 0x55738f4edd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009990 tx=0x7fddf80a9890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bb49fd93037924 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.926+0000 7fddfea00700 1 -- 172.21.15.50:0/831738278 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80aa7a0 con 0x55738f4e3690 2024-05-10T21:10:33.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.926+0000 7fddfea00700 1 -- 172.21.15.50:0/831738278 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf80aa0a0 con 0x55738f4e3690 2024-05-10T21:10:33.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.926+0000 7fddfea00700 1 -- 172.21.15.50:0/831738278 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80ae240 con 0x55738f4e3690 2024-05-10T21:10:33.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 -- 172.21.15.50:0/831738278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 msgr2=0x55738f4edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 0x55738f4edd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009990 tx=0x7fddf80a9890 comp rx=0 tx=0).stop 2024-05-10T21:10:33.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 -- 172.21.15.50:0/831738278 shutdown_connections 2024-05-10T21:10:33.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/831738278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3690 0x55738f4edd50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 -- 172.21.15.50:0/831738278 >> 172.21.15.50:0/831738278 conn(0x55738f3a4c10 msgr2=0x55738f4defb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 -- 172.21.15.50:0/831738278 shutdown_connections 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.927+0000 7fde22a1ab80 1 -- 172.21.15.50:0/831738278 wait complete. 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.928+0000 7fde22a1ab80 1 -- start start 2024-05-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.928+0000 7fde22a1ab80 1 Processor -- start 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3aa0 0x55738f4ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55738f4f09d0 con 0x55738f4e3aa0 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55738f36bcb0 con 0x55738f4e3690 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55738f4f0b50 con 0x55738f365fd0 2024-05-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde22a1ab80 10 client.0 _ll_register_callbacks cb 0x55738f399510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4ed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35476/0 (socket says 172.21.15.50:35476) 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.929+0000 7fde11400700 1 -- 172.21.15.50:0/2043669156 learned_addr learned my addr 172.21.15.50:0/2043669156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.930+0000 7fde10a00700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.930+0000 7fde11e00700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3aa0 0x55738f4ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.931+0000 7fde10a00700 1 -- 172.21.15.50:0/2043669156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 msgr2=0x55738f4ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.931+0000 7fde10a00700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.932+0000 7fde10a00700 1 -- 172.21.15.50:0/2043669156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3aa0 msgr2=0x55738f4ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.932+0000 7fde10a00700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3aa0 0x55738f4ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.932+0000 7fde10a00700 1 -- 172.21.15.50:0/2043669156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738f3cb5b0 con 0x55738f4e3690 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fde10a00700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4edd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fddf80b3030 tx=0x7fddf80096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75229d4c12811c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fddf3400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80b2e70 con 0x55738f4e3690 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fddf80aa300 con 0x55738f4e3690 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf80aa0a0 con 0x55738f4e3690 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.933+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddf80aa7a0 con 0x55738f4e3690 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.934+0000 7fde22a1ab80 10 client.25700 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.934+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55738f4f09d0 con 0x55738f4e3690 2024-05-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.934+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf80aa0a0 con 0x55738f4e3690 2024-05-10T21:10:33.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fddf80a5aa0 con 0x55738f4e3690 2024-05-10T21:10:33.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 10 client.25700 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:33.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 10 client.25700 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:10:33.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 20 client.25700 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f4fd8a0 0x55738f4ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 4 client.25700 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.935+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738f4b0200 con 0x55738f4fd8a0 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.936+0000 7fde11400700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f4fd8a0 0x55738f4ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.937+0000 7fde11400700 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f4fd8a0 0x55738f4ffcc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8f202312d3e2f3 server_cookie=352ed96458c9fe67 in_seq=0 out_seq=0 2024-05-10T21:10:33.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.937+0000 7fddf3400700 10 client.25700 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.937+0000 7fddf3400700 1 -- 172.21.15.50:0/2043669156 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55738f4b0200 con 0x55738f4fd8a0 2024-05-10T21:10:33.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.937+0000 7fddf3400700 10 client.25700 handle_command_reply: tid=0 2024-05-10T21:10:33.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 client.25700 shutdown 2024-05-10T21:10:33.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 msgr2=0x55738f4edd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4edd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fddf80b3030 tx=0x7fddf80096e0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 shutdown_connections 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f4fd8a0 0x55738f4ffcc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3aa0 0x55738f4ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2043669156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4edd40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 >> 172.21.15.50:0/2043669156 conn(0x55738f3a4c10 msgr2=0x55738f4de270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.938+0000 7fddf3400700 0 client.25700 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.939+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 shutdown_connections 2024-05-10T21:10:33.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.939+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2043669156 wait complete. 2024-05-10T21:10:33.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.941+0000 7fde22a1ab80 20 client.25700 trim_cache size 0 max 16384 2024-05-10T21:10:33.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.963+0000 7fde22a1ab80 1 Processor -- start 2024-05-10T21:10:33.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.963+0000 7fde22a1ab80 1 -- start start 2024-05-10T21:10:33.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 0x55738f4f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1af0d0 con 0x55738f365fd0 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ac960 con 0x55738f4e3690 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.964+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c10eff0 con 0x55738f4e3aa0 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 0x55738f4f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43546/0 (socket says 172.21.15.50:43546) 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 -- 172.21.15.50:0/212401439 learned_addr learned my addr 172.21.15.50:0/212401439 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde11e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 0x55738f4f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35490/0 (socket says 172.21.15.50:35490) 2024-05-10T21:10:33.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde11400700 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 -- 172.21.15.50:0/212401439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 msgr2=0x55738f4f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 0x55738f4f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 -- 172.21.15.50:0/212401439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 msgr2=0x55738f4f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde10a00700 1 -- 172.21.15.50:0/212401439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738f4d83b0 con 0x55738f4e3690 2024-05-10T21:10:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.965+0000 7fde11400700 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.966+0000 7fde10a00700 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fddf80a7310 tx=0x7fddf80a90d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7c968bfd109b1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.966+0000 7fddfe000700 1 -- 172.21.15.50:0/212401439 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80b2e70 con 0x55738f4e3690 2024-05-10T21:10:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.966+0000 7fddfe000700 1 -- 172.21.15.50:0/212401439 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf80aa0a0 con 0x55738f4e3690 2024-05-10T21:10:33.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.966+0000 7fddfe000700 1 -- 172.21.15.50:0/212401439 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80b2c00 con 0x55738f4e3690 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 -- 172.21.15.50:0/212401439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 msgr2=0x55738f4f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fddf80a7310 tx=0x7fddf80a90d0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 -- 172.21.15.50:0/212401439 shutdown_connections 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f4e3aa0 0x55738f4f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f365fd0 0x55738f4f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/212401439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3690 0x55738f4f5da0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.967+0000 7fde22a1ab80 1 -- 172.21.15.50:0/212401439 >> 172.21.15.50:0/212401439 conn(0x55738f3a4c10 msgr2=0x55738f4eed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.968+0000 7fde22a1ab80 1 -- 172.21.15.50:0/212401439 shutdown_connections 2024-05-10T21:10:33.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.968+0000 7fde22a1ab80 1 -- 172.21.15.50:0/212401439 wait complete. 2024-05-10T21:10:33.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.968+0000 7fde22a1ab80 1 -- start start 2024-05-10T21:10:33.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 Processor -- start 2024-05-10T21:10:33.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3690 0x55738f4f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1aae80 con 0x55738f4e3690 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ac290 con 0x55738f4e3aa0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ab4f0 con 0x55738f365fd0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.969+0000 7fde22a1ab80 10 client.0 _ll_register_callbacks cb 0x55738f399510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35500/0 (socket says 172.21.15.50:35500) 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 -- 172.21.15.50:0/2516441074 learned_addr learned my addr 172.21.15.50:0/2516441074 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde10a00700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3690 0x55738f4f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11e00700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 -- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 msgr2=0x55738f4f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 -- 172.21.15.50:0/2516441074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3690 msgr2=0x55738f4f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3690 0x55738f4f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.970+0000 7fde11400700 1 -- 172.21.15.50:0/2516441074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738f4f4830 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.971+0000 7fde11400700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55738f500260 tx=0x7fde00012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9116fb0e331f1c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.971+0000 7fddf3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.971+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00010e50 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.971+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fde00004830 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.971+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf80aa0a0 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde00030050 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fde22a1ab80 10 client.25684 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55738f568900 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf80aa0a0 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fde000200b0 con 0x55738f365fd0 2024-05-10T21:10:33.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.972+0000 7fde22a1ab80 10 client.25684 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:33.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.973+0000 7fde22a1ab80 10 client.25684 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:10:33.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.973+0000 7fde22a1ab80 20 client.25684 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:33.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.973+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f5690b0 0x55738f56b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:33.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.973+0000 7fde22a1ab80 4 client.25684 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.35, "await_for": 42.0}] 2024-05-10T21:10:33.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.973+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.35, "await_for": 42.0}) -- 0x55738f413590 con 0x55738f5690b0 2024-05-10T21:10:33.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.974+0000 7fde10a00700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f5690b0 0x55738f56b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:33.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.975+0000 7fde10a00700 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f5690b0 0x55738f56b4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3a2ae856c718fe server_cookie=2e846ce0b5ff0d68 in_seq=0 out_seq=0 2024-05-10T21:10:33.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.975+0000 7fddf3400700 10 client.25684 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.982+0000 7fddf3400700 1 -- 172.21.15.50:0/2516441074 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55738f413590 con 0x55738f5690b0 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.982+0000 7fddf3400700 10 client.25684 handle_command_reply: tid=0 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.982+0000 7fde22a1ab80 1 client.25684 shutdown 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.982+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 msgr2=0x55738f4f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.982+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55738f500260 tx=0x7fde00012730 comp rx=0 tx=0).stop 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 shutdown_connections 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55738f5690b0 0x55738f56b4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55738f4e3aa0 0x55738f4f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55738f365fd0 0x55738f4f5c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 --2- 172.21.15.50:0/2516441074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55738f4e3690 0x55738f4f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fddf3400700 0 client.25684 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:33.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 >> 172.21.15.50:0/2516441074 conn(0x55738f3a4c10 msgr2=0x55738f4ef6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 shutdown_connections 2024-05-10T21:10:33.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.983+0000 7fde22a1ab80 1 -- 172.21.15.50:0/2516441074 wait complete. 2024-05-10T21:10:33.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.984+0000 7fde22a1ab80 20 client.25684 trim_cache size 0 max 16384 2024-05-10T21:10:33.984 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 25, 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 6, 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "31fca0c0": { 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 6, 2024-05-10T21:10:33.985 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:10:33.986 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 81.3, 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:33.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:10:33.988 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:33.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.984+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 msgr2=0x7fddf40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.984+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 0x7fddf40b4e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde04009fd0 tx=0x7fde0400f8e0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.984+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 msgr2=0x7fde0c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:33.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.984+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c19d660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddf80098e0 tx=0x7fddf80055f0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 shutdown_connections 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fddf40b2980 0x7fddf40b4e90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde0c1045f0 0x7fde0c19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c109870 0x7fde0c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 --2- 172.21.15.50:0/2423319615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c108f20 0x7fde0c19d660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.986+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 >> 172.21.15.50:0/2423319615 conn(0x7fde0c100040 msgr2=0x7fde0c110ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:33.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.987+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 shutdown_connections 2024-05-10T21:10:33.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:33.987+0000 7fde13e00700 1 -- 172.21.15.50:0/2423319615 wait complete. 2024-05-10T21:10:35.054 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:34 smithi050 ceph-mon[29546]: pgmap v511: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:35.275 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '31fca0c0', quiesce took 10.5 seconds. Will release after: 21.35 2024-05-10T21:10:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:34 smithi092 ceph-mon[34415]: pgmap v511: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:35.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:34 smithi142 ceph-mon[34846]: pgmap v511: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 -- 172.21.15.50:0/1934913491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 msgr2=0x7f988410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 --2- 172.21.15.50:0/1934913491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988410e130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98700098f0 tx=0x7f9870009c00 comp rx=0 tx=0).stop 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 -- 172.21.15.50:0/1934913491 shutdown_connections 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 --2- 172.21.15.50:0/1934913491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f988410e700 0x7f9884110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 --2- 172.21.15.50:0/1934913491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9884105290 0x7f98841056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 --2- 172.21.15.50:0/1934913491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988410e130 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.033+0000 7f9883200700 1 -- 172.21.15.50:0/1934913491 >> 172.21.15.50:0/1934913491 conn(0x7f9884078ac0 msgr2=0x7f9884078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.034+0000 7f9883200700 1 -- 172.21.15.50:0/1934913491 shutdown_connections 2024-05-10T21:10:36.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.034+0000 7f9883200700 1 -- 172.21.15.50:0/1934913491 wait complete. 2024-05-10T21:10:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.035+0000 7f9883200700 1 Processor -- start 2024-05-10T21:10:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.035+0000 7f9883200700 1 -- start start 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 0x7f988419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 0x7f9884198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9884112120 con 0x7f988410e700 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9884111fa0 con 0x7f9884105be0 2024-05-10T21:10:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9883200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98841122a0 con 0x7f9884105290 2024-05-10T21:10:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 0x7f988419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 0x7f988419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35516/0 (socket says 172.21.15.50:35516) 2024-05-10T21:10:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.036+0000 7f9881e00700 1 -- 172.21.15.50:0/3263650219 learned_addr learned my addr 172.21.15.50:0/3263650219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9882800700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 0x7f9884198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 -- 172.21.15.50:0/3263650219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 msgr2=0x7f988419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 0x7f988419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 -- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 msgr2=0x7f9884198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 0x7f9884198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9881400700 1 -- 172.21.15.50:0/3263650219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884198ff0 con 0x7f9884105be0 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.037+0000 7f9882800700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 0x7f9884198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.038+0000 7f9881400700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988419ec60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f98700052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.038+0000 7f987aa00700 1 -- 172.21.15.50:0/3263650219 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987001d040 con 0x7f9884105be0 2024-05-10T21:10:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.038+0000 7f987aa00700 1 -- 172.21.15.50:0/3263650219 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9870018aa0 con 0x7f9884105be0 2024-05-10T21:10:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.038+0000 7f987aa00700 1 -- 172.21.15.50:0/3263650219 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870023410 con 0x7f9884105be0 2024-05-10T21:10:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.038+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9884199280 con 0x7f9884105be0 2024-05-10T21:10:36.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.039+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9884199780 con 0x7f9884105be0 2024-05-10T21:10:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.044+0000 7f989770cb80 1 Processor -- start 2024-05-10T21:10:36.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 -- start start 2024-05-10T21:10:36.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c1f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1f7c40 0x55925c1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55925c119a20 con 0x55925c079e40 2024-05-10T21:10:36.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55925c0b7760 con 0x55925c1fda80 2024-05-10T21:10:36.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.045+0000 7f989770cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55925c1159e0 con 0x55925c1f7c40 2024-05-10T21:10:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9882800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9882800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50492/0 (socket says 172.21.15.50:50492) 2024-05-10T21:10:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c1f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9882800700 1 -- 172.21.15.50:0/3733790461 learned_addr learned my addr 172.21.15.50:0/3733790461 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 -- 172.21.15.50:0/3733790461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1f7c40 msgr2=0x55925c1f8050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1f7c40 0x55925c1f8050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 -- 172.21.15.50:0/3733790461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 msgr2=0x55925c1fd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9881e00700 1 -- 172.21.15.50:0/3733790461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925c0668d0 con 0x55925c1fda80 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.046+0000 7f9882800700 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.047+0000 7f987aa00700 1 -- 172.21.15.50:0/3263650219 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9870003680 con 0x7f9884105be0 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.047+0000 7f9881e00700 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c1f7700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9884199a80 tx=0x7f987400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7cfac30e75798 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.047+0000 7f986f400700 1 -- 172.21.15.50:0/3733790461 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874026070 con 0x55925c1fda80 2024-05-10T21:10:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.047+0000 7f986f400700 1 -- 172.21.15.50:0/3733790461 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9874004500 con 0x55925c1fda80 2024-05-10T21:10:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.047+0000 7f986f400700 1 -- 172.21.15.50:0/3733790461 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874003e50 con 0x55925c1fda80 2024-05-10T21:10:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 -- 172.21.15.50:0/3733790461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 msgr2=0x55925c1f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c1f7700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9884199a80 tx=0x7f987400f6a0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 -- 172.21.15.50:0/3733790461 shutdown_connections 2024-05-10T21:10:36.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1f7c40 0x55925c1f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 --2- 172.21.15.50:0/3733790461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c1f7700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 -- 172.21.15.50:0/3733790461 >> 172.21.15.50:0/3733790461 conn(0x55925c0bef70 msgr2=0x55925c1f2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.048+0000 7f989770cb80 1 -- 172.21.15.50:0/3733790461 shutdown_connections 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.049+0000 7f989770cb80 1 -- 172.21.15.50:0/3733790461 wait complete. 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.049+0000 7f987aa00700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 0x7f98680b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.049+0000 7f989770cb80 1 -- start start 2024-05-10T21:10:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.049+0000 7f987aa00700 1 -- 172.21.15.50:0/3263650219 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f98700e2930 con 0x7f9884105be0 2024-05-10T21:10:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.050+0000 7f989770cb80 1 Processor -- start 2024-05-10T21:10:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.050+0000 7f9882800700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 0x7f98680b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.050+0000 7f989770cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.050+0000 7f989770cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c2016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f989770cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 0x55925c201bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f989770cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55925c204490 con 0x55925c1fda80 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f989770cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55925c07fb20 con 0x55925c1f7c40 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f989770cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55925c204610 con 0x55925c079e40 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f989770cb80 10 client.0 _ll_register_callbacks cb 0x55925c0c0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f9881400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c2016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.051+0000 7f9882800700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 0x7f98680b4da0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f987c009d00 tx=0x7f987c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35522/0 (socket says 172.21.15.50:35522) 2024-05-10T21:10:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 -- 172.21.15.50:0/2631489491 learned_addr learned my addr 172.21.15.50:0/2631489491 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9882800700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 0x55925c201bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 -- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 msgr2=0x55925c2016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c2016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 -- 172.21.15.50:0/2631489491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 msgr2=0x55925c201bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 0x55925c201bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 -- 172.21.15.50:0/2631489491 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925c0df420 con 0x55925c079e40 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9882800700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 0x55925c201bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.052+0000 7f9881e00700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f987401c040 tx=0x7f987400fe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eace1d8b58b62c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f9863e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874031060 con 0x55925c079e40 2024-05-10T21:10:36.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9874003a50 con 0x55925c079e40 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9874004500 con 0x55925c079e40 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9874026040 con 0x55925c079e40 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f989770cb80 10 client.25693 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.053+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55925c20f070 con 0x55925c079e40 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9874004500 con 0x55925c079e40 2024-05-10T21:10:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9874004950 con 0x55925c079e40 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 20 client.25693 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c211600 0x55925c213a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 4 client.25693 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.054+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55925c1c41c0 con 0x55925c211600 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.055+0000 7f9881400700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c211600 0x55925c213a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.056+0000 7f9863e00700 10 client.25693 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.056+0000 7f9881400700 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c211600 0x55925c213a20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fb0f232b53aa66 server_cookie=efc05eec3e31fc47 in_seq=0 out_seq=0 2024-05-10T21:10:36.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.056+0000 7f9863e00700 1 -- 172.21.15.50:0/2631489491 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55925c1c41c0 con 0x55925c211600 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.056+0000 7f9863e00700 10 client.25693 handle_command_reply: tid=0 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 client.25693 shutdown 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 msgr2=0x55925c201170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f987401c040 tx=0x7f987400fe50 comp rx=0 tx=0).stop 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 shutdown_connections 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c211600 0x55925c213a20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1fda80 0x55925c201bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c201170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 --2- 172.21.15.50:0/2631489491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c2016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 >> 172.21.15.50:0/2631489491 conn(0x55925c0bef70 msgr2=0x55925c1f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f9863e00700 0 client.25693 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:36.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.057+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 shutdown_connections 2024-05-10T21:10:36.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.058+0000 7f989770cb80 1 -- 172.21.15.50:0/2631489491 wait complete. 2024-05-10T21:10:36.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.058+0000 7f989770cb80 20 client.25693 trim_cache size 0 max 16384 2024-05-10T21:10:36.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.074+0000 7f989770cb80 1 Processor -- start 2024-05-10T21:10:36.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.074+0000 7f989770cb80 1 -- start start 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c20d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1fda80 0x55925c1eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aefd0 con 0x55925c079e40 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98841ac860 con 0x55925c1f7c40 2024-05-10T21:10:36.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f989770cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aa6f0 con 0x55925c1fda80 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f9881e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f9881400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c20d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f9881e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50502/0 (socket says 172.21.15.50:50502) 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f9881e00700 1 -- 172.21.15.50:0/3506057236 learned_addr learned my addr 172.21.15.50:0/3506057236 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.075+0000 7f9882800700 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1fda80 0x55925c1eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 -- 172.21.15.50:0/3506057236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1fda80 msgr2=0x55925c1eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1fda80 0x55925c1eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 -- 172.21.15.50:0/3506057236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 msgr2=0x55925c1f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 -- 172.21.15.50:0/3506057236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925c1eec40 con 0x55925c1f7c40 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.076+0000 7f9881400700 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c20d910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98700ab6c0 tx=0x7f9870009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2bc76957b600dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.077+0000 7f986ea00700 1 -- 172.21.15.50:0/3506057236 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700aeb50 con 0x55925c1f7c40 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.077+0000 7f986ea00700 1 -- 172.21.15.50:0/3506057236 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98700ae450 con 0x55925c1f7c40 2024-05-10T21:10:36.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.077+0000 7f986ea00700 1 -- 172.21.15.50:0/3506057236 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700ae7f0 con 0x55925c1f7c40 2024-05-10T21:10:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 -- 172.21.15.50:0/3506057236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 msgr2=0x55925c20d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c20d910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98700ab6c0 tx=0x7f9870009f50 comp rx=0 tx=0).stop 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 -- 172.21.15.50:0/3506057236 shutdown_connections 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c1fda80 0x55925c1eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c079e40 0x55925c1f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 --2- 172.21.15.50:0/3506057236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1f7c40 0x55925c20d910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 -- 172.21.15.50:0/3506057236 >> 172.21.15.50:0/3506057236 conn(0x55925c0bef70 msgr2=0x55925c1fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 -- 172.21.15.50:0/3506057236 shutdown_connections 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.078+0000 7f989770cb80 1 -- 172.21.15.50:0/3506057236 wait complete. 2024-05-10T21:10:36.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.079+0000 7f989770cb80 1 -- start start 2024-05-10T21:10:36.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.079+0000 7f989770cb80 1 Processor -- start 2024-05-10T21:10:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 0x55925c2142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c214820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98841aad40 con 0x55925c1f7c40 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98841ac190 con 0x55925c1fda80 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98841ab3f0 con 0x55925c079e40 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f989770cb80 10 client.0 _ll_register_callbacks cb 0x55925c0c0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f9881e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35536/0 (socket says 172.21.15.50:35536) 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.080+0000 7f9881e00700 1 -- 172.21.15.50:0/2369308303 learned_addr learned my addr 172.21.15.50:0/2369308303 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9882800700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c214820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881400700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 0x55925c2142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881e00700 1 -- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 msgr2=0x55925c214820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881e00700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c214820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881e00700 1 -- 172.21.15.50:0/2369308303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 msgr2=0x55925c2142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881e00700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 0x55925c2142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881e00700 1 -- 172.21.15.50:0/2369308303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925c214f00 con 0x55925c079e40 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.081+0000 7f9881400700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 0x55925c2142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.082+0000 7f9882800700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c214820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.082+0000 7f9881e00700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cb0 tx=0x7f9874003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a8fc23dbe1c3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.082+0000 7f9863e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987403b080 con 0x55925c079e40 2024-05-10T21:10:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98700ae450 con 0x55925c079e40 2024-05-10T21:10:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f987402c900 con 0x55925c079e40 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f987402c470 con 0x55925c079e40 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f989770cb80 10 client.25699 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.083+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55925c27b9a0 con 0x55925c079e40 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98700ae450 con 0x55925c079e40 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9874005550 con 0x55925c079e40 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 10 client.25699 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 20 client.25699 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c27bfb0 0x55925c27e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 4 client.25699 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:36.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.084+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55925c127400 con 0x55925c27bfb0 2024-05-10T21:10:36.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.085+0000 7f9881400700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c27bfb0 0x55925c27e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.085+0000 7f9863e00700 10 client.25699 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.085+0000 7f9881400700 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c27bfb0 0x55925c27e3d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a78c25b3286c80 server_cookie=9da58cafc8db2820 in_seq=0 out_seq=0 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f9863e00700 1 -- 172.21.15.50:0/2369308303 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55925c127400 con 0x55925c27bfb0 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f9863e00700 10 client.25699 handle_command_reply: tid=0 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 client.25699 shutdown 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 msgr2=0x55925c216f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cb0 tx=0x7f9874003f80 comp rx=0 tx=0).stop 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 shutdown_connections 2024-05-10T21:10:36.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55925c27bfb0 0x55925c27e3d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55925c1fda80 0x55925c214820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55925c079e40 0x55925c216f10 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cb0 tx=0x7f9874003f80 comp rx=0 tx=0).stop 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 --2- 172.21.15.50:0/2369308303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925c1f7c40 0x55925c2142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 >> 172.21.15.50:0/2369308303 conn(0x55925c0bef70 msgr2=0x55925c1eca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.086+0000 7f9863e00700 0 client.25699 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.087+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 shutdown_connections 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.087+0000 7f989770cb80 1 -- 172.21.15.50:0/2369308303 wait complete. 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.087+0000 7f989770cb80 20 client.25699 trim_cache size 0 max 16384 2024-05-10T21:10:36.087 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 msgr2=0x7f98680b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 0x7f98680b4da0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f987c009d00 tx=0x7f987c009040 comp rx=0 tx=0).stop 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 msgr2=0x7f988419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988419ec60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9870000c00 tx=0x7f98700052f0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 shutdown_connections 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f98680b2890 0x7f98680b4da0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f988410e700 0x7f9884198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9884105290 0x7f988419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 --2- 172.21.15.50:0/3263650219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9884105be0 0x7f988419ec60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:36.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 >> 172.21.15.50:0/3263650219 conn(0x7f9884078ac0 msgr2=0x7f98841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:36.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 shutdown_connections 2024-05-10T21:10:36.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:36.088+0000 7f9883200700 1 -- 172.21.15.50:0/3263650219 wait complete. 2024-05-10T21:10:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:36 smithi092 ceph-mon[34415]: pgmap v512: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:10:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:36 smithi142 ceph-mon[34846]: pgmap v512: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:10:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:36 smithi050 ceph-mon[29546]: pgmap v512: 97 pgs: 97 active+clean; 674 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T21:10:38.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:10:38.024 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:10:38.505 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:38 smithi092 ceph-mon[34415]: pgmap v513: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:38 smithi142 ceph-mon[34846]: pgmap v513: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:39.384 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:38 smithi050 ceph-mon[29546]: pgmap v513: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:10:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 -- 172.21.15.50:0/418330108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38076430 msgr2=0x7efc381106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 --2- 172.21.15.50:0/418330108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38076430 0x7efc381106e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc240098f0 tx=0x7efc24009c00 comp rx=0 tx=0).stop 2024-05-10T21:10:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 -- 172.21.15.50:0/418330108 shutdown_connections 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 --2- 172.21.15.50:0/418330108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38075ae0 0x7efc38075ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 --2- 172.21.15.50:0/418330108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 0x7efc38113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 --2- 172.21.15.50:0/418330108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38076430 0x7efc381106e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 -- 172.21.15.50:0/418330108 >> 172.21.15.50:0/418330108 conn(0x7efc38100040 msgr2=0x7efc38102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 -- 172.21.15.50:0/418330108 shutdown_connections 2024-05-10T21:10:40.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.771+0000 7efc40000700 1 -- 172.21.15.50:0/418330108 wait complete. 2024-05-10T21:10:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.772+0000 7efc40000700 1 Processor -- start 2024-05-10T21:10:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.772+0000 7efc40000700 1 -- start start 2024-05-10T21:10:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38076430 0x7efc3819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 0x7efc381a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc381166d0 con 0x7efc38075ae0 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc38116550 con 0x7efc38076430 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc40000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc38116850 con 0x7efc38110c20 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc3d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc3d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50526/0 (socket says 172.21.15.50:50526) 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc3d600700 1 -- 172.21.15.50:0/3185082960 learned_addr learned my addr 172.21.15.50:0/3185082960 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc3e000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 0x7efc381a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.773+0000 7efc3cc00700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38076430 0x7efc3819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 -- 172.21.15.50:0/3185082960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 msgr2=0x7efc381a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 0x7efc381a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 -- 172.21.15.50:0/3185082960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38076430 msgr2=0x7efc3819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38076430 0x7efc3819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 -- 172.21.15.50:0/3185082960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc381a1fd0 con 0x7efc38075ae0 2024-05-10T21:10:40.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc3d600700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ec10 tx=0x7efc2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc36000700 1 -- 172.21.15.50:0/3185082960 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00cc80 con 0x7efc38075ae0 2024-05-10T21:10:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc36000700 1 -- 172.21.15.50:0/3185082960 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2c00ce20 con 0x7efc38075ae0 2024-05-10T21:10:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.774+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc381a22c0 con 0x7efc38075ae0 2024-05-10T21:10:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.775+0000 7efc36000700 1 -- 172.21.15.50:0/3185082960 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c010430 con 0x7efc38075ae0 2024-05-10T21:10:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.775+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc381a9be0 con 0x7efc38075ae0 2024-05-10T21:10:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.779+0000 7efc36000700 1 -- 172.21.15.50:0/3185082960 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efc2c0105d0 con 0x7efc38075ae0 2024-05-10T21:10:40.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.780+0000 7efc4eabfb80 1 Processor -- start 2024-05-10T21:10:40.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.780+0000 7efc4eabfb80 1 -- start start 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.780+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.780+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae68b0 0x55f581ae0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae0a70 0x55f581ae0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f581a02aa0 con 0x55f581ae0a70 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5819a07e0 con 0x55f581ae68b0 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f5819fea60 con 0x55f581962ec0 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae68b0 0x55f581ae0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35572/0 (socket says 172.21.15.50:35572) 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 -- 172.21.15.50:0/3114049207 learned_addr learned my addr 172.21.15.50:0/3114049207 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3cc00700 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae0a70 0x55f581ae0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 -- 172.21.15.50:0/3114049207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae68b0 msgr2=0x55f581ae0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae68b0 0x55f581ae0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 -- 172.21.15.50:0/3114049207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae0a70 msgr2=0x55f581ae0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae0a70 0x55f581ae0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc3e000700 1 -- 172.21.15.50:0/3114049207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58194f950 con 0x55f581962ec0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.781+0000 7efc36000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 0x7efc200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc36000700 1 -- 172.21.15.50:0/3185082960 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efc2c0e2490 con 0x7efc38075ae0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc3e000700 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc30011630 tx=0x7efc30009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9645e995381583 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc3d600700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 0x7efc200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc2aa00700 1 -- 172.21.15.50:0/3114049207 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30003a00 con 0x55f581962ec0 2024-05-10T21:10:40.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc2aa00700 1 -- 172.21.15.50:0/3114049207 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3001a070 con 0x55f581962ec0 2024-05-10T21:10:40.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.782+0000 7efc2aa00700 1 -- 172.21.15.50:0/3114049207 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30007760 con 0x55f581962ec0 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.783+0000 7efc4eabfb80 1 -- 172.21.15.50:0/3114049207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 msgr2=0x55f581ae6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.783+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc30011630 tx=0x7efc30009920 comp rx=0 tx=0).stop 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.783+0000 7efc3d600700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 0x7efc200b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f581adcdc0 tx=0x7efc2c0e5f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 -- 172.21.15.50:0/3114049207 shutdown_connections 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae0a70 0x55f581ae0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581ae6370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/3114049207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae68b0 0x55f581ae0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 -- 172.21.15.50:0/3114049207 >> 172.21.15.50:0/3114049207 conn(0x55f5819a7ff0 msgr2=0x55f581adb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 -- 172.21.15.50:0/3114049207 shutdown_connections 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.784+0000 7efc4eabfb80 1 -- 172.21.15.50:0/3114049207 wait complete. 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.785+0000 7efc4eabfb80 1 -- start start 2024-05-10T21:10:40.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.785+0000 7efc4eabfb80 1 Processor -- start 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581962ec0 0x55f581aea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aeaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f581aed330 con 0x55f581ae68b0 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f581968ba0 con 0x55f581962ec0 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f581aed4b0 con 0x55f581ae0a70 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc4eabfb80 10 client.0 _ll_register_callbacks cb 0x55f5819a9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc3cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc3cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35580/0 (socket says 172.21.15.50:35580) 2024-05-10T21:10:40.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc3e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581962ec0 0x55f581aea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc3cc00700 1 -- 172.21.15.50:0/685388426 learned_addr learned my addr 172.21.15.50:0/685388426 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.786+0000 7efc3e000700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aeaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 -- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581962ec0 msgr2=0x55f581aea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581962ec0 0x55f581aea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 -- 172.21.15.50:0/685388426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 msgr2=0x55f581aeaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aeaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 -- 172.21.15.50:0/685388426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5819c84a0 con 0x55f581ae0a70 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3e000700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:40.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc3cc00700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc24009810 tx=0x7efc2400f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a17f8f78f6ae77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc1f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:40.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24005240 con 0x55f581ae0a70 2024-05-10T21:10:40.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efc240053e0 con 0x55f581ae0a70 2024-05-10T21:10:40.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.787+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc3001a070 con 0x55f581ae0a70 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc240055e0 con 0x55f581ae0a70 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 10 client.25711 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f581af7db0 con 0x55f581ae0a70 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3001a070 con 0x55f581ae0a70 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efc2400b5a0 con 0x55f581ae0a70 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 10 client.25711 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 20 client.25711 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581afa490 0x55f581afc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 4 client.25711 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:40.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.788+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f581aad240 con 0x55f581afa490 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.789+0000 7efc3e000700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581afa490 0x55f581afc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.789+0000 7efc3e000700 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581afa490 0x55f581afc8b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364be42707b0a344 server_cookie=774bbf3c3ec2ff9a in_seq=0 out_seq=0 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.789+0000 7efc1f400700 10 client.25711 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.790+0000 7efc1f400700 1 -- 172.21.15.50:0/685388426 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f581aad240 con 0x55f581afa490 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.790+0000 7efc1f400700 10 client.25711 handle_command_reply: tid=0 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.790+0000 7efc4eabfb80 1 client.25711 shutdown 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.790+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 msgr2=0x55f581aea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.790+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc24009810 tx=0x7efc2400f670 comp rx=0 tx=0).stop 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 shutdown_connections 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581afa490 0x55f581afc8b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581962ec0 0x55f581aea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/685388426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae0a70 0x55f581aea540 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc1f400700 0 client.25711 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 >> 172.21.15.50:0/685388426 conn(0x55f5819a7ff0 msgr2=0x55f581adf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 shutdown_connections 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 1 -- 172.21.15.50:0/685388426 wait complete. 2024-05-10T21:10:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.791+0000 7efc4eabfb80 20 client.25711 trim_cache size 0 max 16384 2024-05-10T21:10:40.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 Processor -- start 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 -- start start 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 0x55f581ae0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 0x55f581aea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b3750 con 0x55f581962ec0 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b0fe0 con 0x55f581ae0a70 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.825+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc381aee70 con 0x55f581ae68b0 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 0x55f581ae0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43696/0 (socket says 172.21.15.50:43696) 2024-05-10T21:10:40.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 0x55f581ae0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50562/0 (socket says 172.21.15.50:50562) 2024-05-10T21:10:40.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 -- 172.21.15.50:0/4129277672 learned_addr learned my addr 172.21.15.50:0/4129277672 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:40.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 -- 172.21.15.50:0/4129277672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 msgr2=0x55f581aea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3e000700 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 0x55f581aea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 0x55f581aea8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 -- 172.21.15.50:0/4129277672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 msgr2=0x55f581ae0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 0x55f581ae0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3cc00700 1 -- 172.21.15.50:0/4129277672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f581ae6da0 con 0x55f581ae0a70 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.826+0000 7efc3e000700 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 0x55f581aea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.827+0000 7efc3cc00700 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc240094a0 tx=0x7efc24009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76695e682601809 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.827+0000 7efc2a000700 1 -- 172.21.15.50:0/4129277672 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2400bbe0 con 0x55f581ae0a70 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.827+0000 7efc2a000700 1 -- 172.21.15.50:0/4129277672 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2400bd80 con 0x55f581ae0a70 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.827+0000 7efc2a000700 1 -- 172.21.15.50:0/4129277672 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24003a50 con 0x55f581ae0a70 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 -- 172.21.15.50:0/4129277672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 msgr2=0x55f581ae4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc240094a0 tx=0x7efc24009fd0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 -- 172.21.15.50:0/4129277672 shutdown_connections 2024-05-10T21:10:40.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581ae68b0 0x55f581aea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581962ec0 0x55f581ae0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/4129277672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581ae0a70 0x55f581ae4760 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 -- 172.21.15.50:0/4129277672 >> 172.21.15.50:0/4129277672 conn(0x55f5819a7ff0 msgr2=0x55f581ae5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 -- 172.21.15.50:0/4129277672 shutdown_connections 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.828+0000 7efc4eabfb80 1 -- 172.21.15.50:0/4129277672 wait complete. 2024-05-10T21:10:40.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.829+0000 7efc4eabfb80 1 -- start start 2024-05-10T21:10:40.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.829+0000 7efc4eabfb80 1 Processor -- start 2024-05-10T21:10:40.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581aefc10 0x55f581af0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc381af4c0 con 0x55f581ae68b0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efc381b0910 con 0x55f581aefc10 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc381afb70 con 0x55f581962ec0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc4eabfb80 10 client.0 _ll_register_callbacks cb 0x55f5819a9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35600/0 (socket says 172.21.15.50:35600) 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3e000700 1 -- 172.21.15.50:0/1636158021 learned_addr learned my addr 172.21.15.50:0/1636158021 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3cc00700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3e000700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581aefc10 0x55f581af0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.830+0000 7efc3e000700 1 -- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581aefc10 msgr2=0x55f581af0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc3e000700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581aefc10 0x55f581af0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc3e000700 1 -- 172.21.15.50:0/1636158021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 msgr2=0x55f581aef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc3e000700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc3e000700 1 -- 172.21.15.50:0/1636158021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f581b64c70 con 0x55f581962ec0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc3e000700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc30011630 tx=0x7efc30000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5dba57e573007bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:40.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc1f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30004ab0 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc2400bd80 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7efc30004c50 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc30035050 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc4eabfb80 10 client.25714 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.831+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f581b65e60 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2400bd80 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efc3002e570 con 0x55f581962ec0 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 10 client.25714 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 20 client.25714 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:40.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581b66320 0x55f581b68740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:40.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 4 client.25714 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:10:40.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.832+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f581a10480 con 0x55f581b66320 2024-05-10T21:10:40.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.833+0000 7efc3cc00700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581b66320 0x55f581b68740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.833+0000 7efc3cc00700 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581b66320 0x55f581b68740 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad83c8d317dec52a server_cookie=73e06f36d7cb0a01 in_seq=0 out_seq=0 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.833+0000 7efc1f400700 10 client.25714 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.834+0000 7efc1f400700 1 -- 172.21.15.50:0/1636158021 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f581a10480 con 0x55f581b66320 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.834+0000 7efc1f400700 10 client.25714 handle_command_reply: tid=0 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.834+0000 7efc4eabfb80 1 client.25714 shutdown 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 msgr2=0x55f581af2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc30011630 tx=0x7efc30000f40 comp rx=0 tx=0).stop 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 shutdown_connections 2024-05-10T21:10:40.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f581b66320 0x55f581b68740 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f581aefc10 0x55f581af0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581962ec0 0x55f581af2c10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 --2- 172.21.15.50:0/1636158021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f581ae68b0 0x55f581aef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 >> 172.21.15.50:0/1636158021 conn(0x55f5819a7ff0 msgr2=0x55f581ae5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc1f400700 0 client.25714 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 shutdown_connections 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 1 -- 172.21.15.50:0/1636158021 wait complete. 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.835+0000 7efc4eabfb80 20 client.25714 trim_cache size 0 max 16384 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:40.836 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "a7e88de8-8123-412f-909a-e11efad06694", 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.836+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 msgr2=0x7efc200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.836+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 0x7efc200b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f581adcdc0 tx=0x7efc2c0e5f70 comp rx=0 tx=0).stop 2024-05-10T21:10:40.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.836+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 msgr2=0x7efc3819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.836+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ec10 tx=0x7efc2c00ef20 comp rx=0 tx=0).stop 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 shutdown_connections 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7efc200b2880 0x7efc200b4d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc38110c20 0x7efc381a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc38075ae0 0x7efc3819d100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 --2- 172.21.15.50:0/3185082960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc38076430 0x7efc3819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 >> 172.21.15.50:0/3185082960 conn(0x7efc38100040 msgr2=0x7efc38101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 shutdown_connections 2024-05-10T21:10:40.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:40.837+0000 7efc40000700 1 -- 172.21.15.50:0/3185082960 wait complete. 2024-05-10T21:10:41.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:40 smithi050 ceph-mon[29546]: pgmap v514: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:41.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:40 smithi092 ceph-mon[34415]: pgmap v514: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:40 smithi142 ceph-mon[34846]: pgmap v514: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:10:41.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:10:42.351 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:42.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:41 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:10:42.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:41 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:42.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:41 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:10:42.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:41 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:41 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:10:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:41 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:41 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:10:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:41 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:42.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:41 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:10:42.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:41 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:42.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:41 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:10:42.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:41 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:10:43.123 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:42 smithi050 ceph-mon[29546]: pgmap v515: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:43.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:42 smithi092 ceph-mon[34415]: pgmap v515: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:42 smithi142 ceph-mon[34846]: pgmap v515: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:43.510 INFO:tasks.workunit.client.0.smithi050.stderr:Updating files: 58% (7227/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 94% (11552/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T21:10:44.499 INFO:tasks.workunit.client.0.smithi050.stderr:Note: switching to '7b710596db3f44de0a4b72eff459db857656fe05'. 2024-05-10T21:10:44.499 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.499 INFO:tasks.workunit.client.0.smithi050.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: git switch -c 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:Or undo this operation with: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: git switch - 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:10:44.500 INFO:tasks.workunit.client.0.smithi050.stderr:HEAD is now at 7b710596db3 Merge PR #56052 into wip-vshankar-testing-20240509.114851-debug 2024-05-10T21:10:44.516 DEBUG:teuthology.orchestra.run.smithi050:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T21:10:44.604 INFO:tasks.workunit.client.0.smithi050.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T21:10:44.612 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T21:10:44.612 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T21:10:44.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 -- 172.21.15.50:0/4265847699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 msgr2=0x7f75bc113380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 --2- 172.21.15.50:0/4265847699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc113380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b0a0 tx=0x7f75b400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 -- 172.21.15.50:0/4265847699 shutdown_connections 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 --2- 172.21.15.50:0/4265847699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc113380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 --2- 172.21.15.50:0/4265847699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 0x7f75bc10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 --2- 172.21.15.50:0/4265847699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc1069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 -- 172.21.15.50:0/4265847699 >> 172.21.15.50:0/4265847699 conn(0x7f75bc102080 msgr2=0x7f75bc1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 -- 172.21.15.50:0/4265847699 shutdown_connections 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.720+0000 7f75c4200700 1 -- 172.21.15.50:0/4265847699 wait complete. 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.721+0000 7f75c4200700 1 Processor -- start 2024-05-10T21:10:44.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 -- start start 2024-05-10T21:10:44.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 0x7f75bc1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc116710 con 0x7f75bc106ef0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc116590 con 0x7f75bc1065a0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.722+0000 7f75c4200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc116890 con 0x7f75bc10c5e0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c0e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49030/0 (socket says 172.21.15.50:49030) 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 -- 172.21.15.50:0/3658788778 learned_addr learned my addr 172.21.15.50:0/3658788778 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c2200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 0x7f75bc1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 -- 172.21.15.50:0/3658788778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 msgr2=0x7f75bc1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 0x7f75bc1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 -- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 msgr2=0x7f75bc19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 -- 172.21.15.50:0/3658788778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc1a1e90 con 0x7f75bc1065a0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.723+0000 7f75c1800700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f75b000e970 tx=0x7f75b000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.724+0000 7f75ba000700 1 -- 172.21.15.50:0/3658788778 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004a30 con 0x7f75bc1065a0 2024-05-10T21:10:44.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.724+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc1a2180 con 0x7f75bc1065a0 2024-05-10T21:10:44.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.724+0000 7f75ba000700 1 -- 172.21.15.50:0/3658788778 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b000cc90 con 0x7f75bc1065a0 2024-05-10T21:10:44.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.724+0000 7f75ba000700 1 -- 172.21.15.50:0/3658788778 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0005610 con 0x7f75bc1065a0 2024-05-10T21:10:44.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.724+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc1a9aa0 con 0x7f75bc1065a0 2024-05-10T21:10:44.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.728+0000 7f75ba000700 1 -- 172.21.15.50:0/3658788778 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f75b000c7e0 con 0x7f75bc1065a0 2024-05-10T21:10:44.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 Processor -- start 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 -- start start 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe55060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5622cbd76a20 con 0x5622cbcd6e40 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5622cbd14760 con 0x5622cbe54c50 2024-05-10T21:10:44.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.729+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5622cbd729e0 con 0x5622cbe5abf0 2024-05-10T21:10:44.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47584/0 (socket says 172.21.15.50:47584) 2024-05-10T21:10:44.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 -- 172.21.15.50:0/2042409324 learned_addr learned my addr 172.21.15.50:0/2042409324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c0e00700 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe55060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 -- 172.21.15.50:0/2042409324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 msgr2=0x5622cbe54710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe54710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 -- 172.21.15.50:0/2042409324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 msgr2=0x5622cbe55060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe55060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75c2200700 1 -- 172.21.15.50:0/2042409324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cbcc38d0 con 0x5622cbcd6e40 2024-05-10T21:10:44.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.730+0000 7f75ba000700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 0x7f75a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75ba000700 1 -- 172.21.15.50:0/3658788778 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f75b00e22d0 con 0x7f75bc1065a0 2024-05-10T21:10:44.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75c2200700 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75b4015000 tx=0x7f75b40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3297cf65cf03dcae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75c1800700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 0x7f75a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75aea00700 1 -- 172.21.15.50:0/2042409324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400bcb0 con 0x5622cbcd6e40 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75aea00700 1 -- 172.21.15.50:0/2042409324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b4006020 con 0x5622cbcd6e40 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.731+0000 7f75aea00700 1 -- 172.21.15.50:0/2042409324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4005040 con 0x5622cbcd6e40 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2042409324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 msgr2=0x5622cbe5a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75b4015000 tx=0x7f75b40092e0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75c1800700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 0x7f75a40b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75b00a9620 tx=0x7f75b00a5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2042409324 shutdown_connections 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe55060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe5a6b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2042409324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2042409324 >> 172.21.15.50:0/2042409324 conn(0x5622cbd15a80 msgr2=0x5622cbe4f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2042409324 shutdown_connections 2024-05-10T21:10:44.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.732+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2042409324 wait complete. 2024-05-10T21:10:44.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.733+0000 7f75d2cf1b80 1 -- start start 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.733+0000 7f75d2cf1b80 1 Processor -- start 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 0x5622cbe5e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe5abf0 0x5622cbe5edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5622cbe61660 con 0x5622cbe54c50 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5622cbcdcb20 con 0x5622cbe5abf0 2024-05-10T21:10:44.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5622cbe617e0 con 0x5622cbcd6e40 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75d2cf1b80 10 client.0 _ll_register_callbacks cb 0x5622cbd0a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c2200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 0x5622cbe5e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c0e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c2200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 0x5622cbe5e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58736/0 (socket says 172.21.15.50:58736) 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c0e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47586/0 (socket says 172.21.15.50:47586) 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c2200700 1 -- 172.21.15.50:0/3825802422 learned_addr learned my addr 172.21.15.50:0/3825802422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.734+0000 7f75c2200700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe5abf0 0x5622cbe5edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 -- 172.21.15.50:0/3825802422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 msgr2=0x5622cbe5e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 0x5622cbe5e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 -- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe5abf0 msgr2=0x5622cbe5edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe5abf0 0x5622cbe5edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 -- 172.21.15.50:0/3825802422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cbd3c420 con 0x5622cbe54c50 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75c0e00700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75bc06acf0 tx=0x7f75a8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46683d3eb179fe62 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75a3400700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:44.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.735+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8004300 con 0x5622cbe54c50 2024-05-10T21:10:44.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f75a80044a0 con 0x5622cbe54c50 2024-05-10T21:10:44.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b4006020 con 0x5622cbe54c50 2024-05-10T21:10:44.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75a8022070 con 0x5622cbe54c50 2024-05-10T21:10:44.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75d2cf1b80 10 client.15627 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:44.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622cbe6c270 con 0x5622cbe54c50 2024-05-10T21:10:44.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b4006020 con 0x5622cbe54c50 2024-05-10T21:10:44.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.736+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f75a801b860 con 0x5622cbe54c50 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 20 client.15627 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbe6e900 0x5622cbe70d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 4 client.15627 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622cbe21070 con 0x5622cbe6e900 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.737+0000 7f75c2200700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbe6e900 0x5622cbe70d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.738+0000 7f75c2200700 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbe6e900 0x5622cbe70d50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f24f2eaa90ad46 server_cookie=6da2047f19240269 in_seq=0 out_seq=0 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.738+0000 7f75a3400700 10 client.15627 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.738+0000 7f75a3400700 1 -- 172.21.15.50:0/3825802422 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622cbe21070 con 0x5622cbe6e900 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.738+0000 7f75a3400700 10 client.15627 handle_command_reply: tid=0 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.738+0000 7f75d2cf1b80 1 client.15627 shutdown 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 msgr2=0x5622cbe5e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75bc06acf0 tx=0x7f75a8011720 comp rx=0 tx=0).stop 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 shutdown_connections 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbe6e900 0x5622cbe70d50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe5abf0 0x5622cbe5edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbcd6e40 0x5622cbe5e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/3825802422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5e8b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75a3400700 0 client.15627 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 >> 172.21.15.50:0/3825802422 conn(0x5622cbd15a80 msgr2=0x5622cbe49430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 shutdown_connections 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/3825802422 wait complete. 2024-05-10T21:10:44.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.739+0000 7f75d2cf1b80 20 client.15627 trim_cache size 0 max 16384 2024-05-10T21:10:44.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.761+0000 7f75d2cf1b80 1 Processor -- start 2024-05-10T21:10:44.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.761+0000 7f75d2cf1b80 1 -- start start 2024-05-10T21:10:44.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 0x5622cbe54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe4f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b3540 con 0x5622cbe54c50 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b0dd0 con 0x5622cbcd6e40 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1aec40 con 0x5622cbe5abf0 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75c0e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75c0e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47590/0 (socket says 172.21.15.50:47590) 2024-05-10T21:10:44.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75c0e00700 1 -- 172.21.15.50:0/898056048 learned_addr learned my addr 172.21.15.50:0/898056048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75c2200700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 0x5622cbe54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.762+0000 7f75c2200700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe4f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 -- 172.21.15.50:0/898056048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 msgr2=0x5622cbe4f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe4f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 -- 172.21.15.50:0/898056048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 msgr2=0x5622cbe54710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 0x5622cbe54710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 -- 172.21.15.50:0/898056048 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cbe55930 con 0x5622cbe54c50 2024-05-10T21:10:44.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c2200700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 0x5622cbe54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:44.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75c0e00700 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f75a8025d40 tx=0x7f75a8025d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12865a041ae665cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75ae000700 1 -- 172.21.15.50:0/898056048 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8005380 con 0x5622cbe54c50 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.763+0000 7f75ae000700 1 -- 172.21.15.50:0/898056048 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a801b8d0 con 0x5622cbe54c50 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75ae000700 1 -- 172.21.15.50:0/898056048 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003940 con 0x5622cbe54c50 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/898056048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 msgr2=0x5622cbe5ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f75a8025d40 tx=0x7f75a8025d70 comp rx=0 tx=0).stop 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/898056048 shutdown_connections 2024-05-10T21:10:44.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbcd6e40 0x5622cbe54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe4f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.764+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/898056048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbe54c50 0x5622cbe5ec30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.765+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/898056048 >> 172.21.15.50:0/898056048 conn(0x5622cbd15a80 msgr2=0x5622cbe4ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.765+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/898056048 shutdown_connections 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.765+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/898056048 wait complete. 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.765+0000 7f75d2cf1b80 1 -- start start 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.765+0000 7f75d2cf1b80 1 Processor -- start 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe66fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe63a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1af290 con 0x5622cbcd6e40 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1b0700 con 0x5622cbe54c50 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1af960 con 0x5622cbe5abf0 2024-05-10T21:10:44.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75d2cf1b80 10 client.0 _ll_register_callbacks cb 0x5622cbd0a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75c2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75c2200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47596/0 (socket says 172.21.15.50:47596) 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75c2200700 1 -- 172.21.15.50:0/2071476245 learned_addr learned my addr 172.21.15.50:0/2071476245 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.766+0000 7f75c0e00700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe66fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe63a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 -- 172.21.15.50:0/2071476245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 msgr2=0x5622cbe63a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe63a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 -- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 msgr2=0x5622cbe66fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe66fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 -- 172.21.15.50:0/2071476245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cbe63f80 con 0x5622cbcd6e40 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c0e00700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe66fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75c2200700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75b4007b10 tx=0x7f75b4028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f652c8d910b8e887 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75a3400700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:44.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b403e060 con 0x5622cbcd6e40 2024-05-10T21:10:44.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f75b400bab0 con 0x5622cbcd6e40 2024-05-10T21:10:44.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.767+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a801b8d0 con 0x5622cbcd6e40 2024-05-10T21:10:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75b400bcb0 con 0x5622cbcd6e40 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 10 client.15633 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622cbed97a0 con 0x5622cbcd6e40 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a801b8d0 con 0x5622cbcd6e40 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f75b4011070 con 0x5622cbcd6e40 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 20 client.15633 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbed9a20 0x5622cbedbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 4 client.15633 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.768+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622cbd84400 con 0x5622cbed9a20 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.769+0000 7f75c0e00700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbed9a20 0x5622cbedbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.769+0000 7f75c0e00700 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbed9a20 0x5622cbedbe40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc78a230b61f144 server_cookie=64c628d2082cf251 in_seq=0 out_seq=0 2024-05-10T21:10:44.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.769+0000 7f75a3400700 10 client.15633 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.769+0000 7f75a3400700 1 -- 172.21.15.50:0/2071476245 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622cbd84400 con 0x5622cbed9a20 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.769+0000 7f75a3400700 10 client.15633 handle_command_reply: tid=0 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 client.15633 shutdown 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 msgr2=0x5622cbe66a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75b4007b10 tx=0x7f75b4028040 comp rx=0 tx=0).stop 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 shutdown_connections 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5622cbed9a20 0x5622cbedbe40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5622cbe5abf0 0x5622cbe63a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5622cbcd6e40 0x5622cbe66a70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 --2- 172.21.15.50:0/2071476245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622cbe54c50 0x5622cbe66fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75a3400700 0 client.15633 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 >> 172.21.15.50:0/2071476245 conn(0x5622cbd15a80 msgr2=0x5622cbe58860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 shutdown_connections 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 1 -- 172.21.15.50:0/2071476245 wait complete. 2024-05-10T21:10:44.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.770+0000 7f75d2cf1b80 20 client.15633 trim_cache size 0 max 16384 2024-05-10T21:10:44.773 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:10:44.773 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:10:44.773 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:10:44.773 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 msgr2=0x7f75a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 0x7f75a40b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75b00a9620 tx=0x7f75b00a5040 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 msgr2=0x7f75bc19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f75b000e970 tx=0x7f75b000ec80 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 shutdown_connections 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f75a40b2880 0x7f75a40b4d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75bc1065a0 0x7f75bc19cfc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75bc10c5e0 0x7f75bc1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 --2- 172.21.15.50:0/3658788778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75bc106ef0 0x7f75bc19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:44.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 >> 172.21.15.50:0/3658788778 conn(0x7f75bc102080 msgr2=0x7f75bc1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:44.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 shutdown_connections 2024-05-10T21:10:44.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:44.771+0000 7f75c4200700 1 -- 172.21.15.50:0/3658788778 wait complete. 2024-05-10T21:10:45.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:44 smithi050 ceph-mon[29546]: pgmap v516: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:45.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:44 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:10:45.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:44 smithi092 ceph-mon[34415]: pgmap v516: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:45.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:44 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:10:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:44 smithi142 ceph-mon[34846]: pgmap v516: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T21:10:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:44 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:10:45.892 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:10:45.892 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:10:46.233 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T21:10:46.340 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T21:10:46.358 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:46.436 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T21:10:46.441 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T21:10:46.441 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T21:10:46.523 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T21:10:46.532 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:10:46.532 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T21:10:46.587 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-05-10T21:10:46.589 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-05-10T21:10:46.589 DEBUG:teuthology.orchestra.run.smithi050:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b710596db3f44de0a4b72eff459db857656fe05 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-05-10T21:10:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 -- 172.21.15.50:0/429659003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 msgr2=0x7f921010bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 --2- 172.21.15.50:0/429659003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f921010bef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9200009900 tx=0x7f920002a3f0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 -- 172.21.15.50:0/429659003 shutdown_connections 2024-05-10T21:10:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 --2- 172.21.15.50:0/429659003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f921010bef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 --2- 172.21.15.50:0/429659003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f92101051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 --2- 172.21.15.50:0/429659003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 0x7f9210104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 -- 172.21.15.50:0/429659003 >> 172.21.15.50:0/429659003 conn(0x7f92100fed80 msgr2=0x7f9210101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 -- 172.21.15.50:0/429659003 shutdown_connections 2024-05-10T21:10:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.094+0000 7f9216200640 1 -- 172.21.15.50:0/429659003 wait complete. 2024-05-10T21:10:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 Processor -- start 2024-05-10T21:10:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 -- start start 2024-05-10T21:10:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 0x7f921019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f92101a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f921010f5e0 con 0x7f9210109ad0 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f921010f460 con 0x7f9210104dd0 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.096+0000 7f9216200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f921010f760 con 0x7f9210103b90 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f9215800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f92101a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49086/0 (socket says 172.21.15.50:49086) 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 -- 172.21.15.50:0/1278534141 learned_addr learned my addr 172.21.15.50:0/1278534141 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f9214e00640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 0x7f921019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 -- 172.21.15.50:0/1278534141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 msgr2=0x7f921019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 0x7f921019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 -- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 msgr2=0x7f92101a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f92101a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 -- 172.21.15.50:0/1278534141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92101a3ed0 con 0x7f9210104dd0 2024-05-10T21:10:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.097+0000 7f920fe00640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91f800e9f0 tx=0x7f91f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.098+0000 7f920d600640 1 -- 172.21.15.50:0/1278534141 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800cd80 con 0x7f9210104dd0 2024-05-10T21:10:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.098+0000 7f920d600640 1 -- 172.21.15.50:0/1278534141 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f80108e0 con 0x7f9210104dd0 2024-05-10T21:10:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.098+0000 7f920d600640 1 -- 172.21.15.50:0/1278534141 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018980 con 0x7f9210104dd0 2024-05-10T21:10:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.098+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92101a41c0 con 0x7f9210104dd0 2024-05-10T21:10:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.098+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101a4710 con 0x7f9210104dd0 2024-05-10T21:10:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.102+0000 7f921e856740 1 Processor -- start 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 -- start start 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c3b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c42780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c04b9f550 con 0x558c048702f0 2024-05-10T21:10:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c04b44bc0 con 0x558c048991b0 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f921e856740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558c04b9b510 con 0x558c048784d0 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.103+0000 7f920d600640 1 -- 172.21.15.50:0/1278534141 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f91f8010430 con 0x7f9210104dd0 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9215800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c3b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47626/0 (socket says 172.21.15.50:47626) 2024-05-10T21:10:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 -- 172.21.15.50:0/3667933867 learned_addr learned my addr 172.21.15.50:0/3667933867 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f920fe00640 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c42780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 -- 172.21.15.50:0/3667933867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 msgr2=0x558c04c42780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c42780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 -- 172.21.15.50:0/3667933867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 msgr2=0x558c04c3b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c3b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 -- 172.21.15.50:0/3667933867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c048b1210 con 0x558c048702f0 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.104+0000 7f9214e00640 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9208009fd0 tx=0x7f92080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed22e0afe512c841 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.105+0000 7f9206000640 1 -- 172.21.15.50:0/3667933867 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208004b20 con 0x558c048702f0 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.105+0000 7f9206000640 1 -- 172.21.15.50:0/3667933867 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9208003e20 con 0x558c048702f0 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.105+0000 7f9206000640 1 -- 172.21.15.50:0/3667933867 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920800f5a0 con 0x558c048702f0 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.105+0000 7f920d600640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 0x7f91f40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f920d600640 1 -- 172.21.15.50:0/1278534141 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f91f80e2560 con 0x7f9210104dd0 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f9215800640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 0x7f91f40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 -- 172.21.15.50:0/3667933867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 msgr2=0x558c04c3b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9208009fd0 tx=0x7f92080118d0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 -- 172.21.15.50:0/3667933867 shutdown_connections 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c42780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048702f0 0x558c04c3b880 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 --2- 172.21.15.50:0/3667933867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c3b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 -- 172.21.15.50:0/3667933867 >> 172.21.15.50:0/3667933867 conn(0x558c0488d300 msgr2=0x558c04c36c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 -- 172.21.15.50:0/3667933867 shutdown_connections 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.106+0000 7f921e856740 1 -- 172.21.15.50:0/3667933867 wait complete. 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.107+0000 7f9215800640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 0x7f91f40b4cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9200031440 tx=0x7f92000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.107+0000 7f921e856740 1 -- start start 2024-05-10T21:10:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.107+0000 7f921e856740 1 Processor -- start 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.107+0000 7f921e856740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c4c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c04c431d0 con 0x558c048784d0 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c04b58660 con 0x558c048991b0 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558c04c43350 con 0x558c048702f0 2024-05-10T21:10:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f921e856740 10 client.0 _ll_register_callbacks cb 0x558c04b22400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f920fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58802/0 (socket says 172.21.15.50:58802) 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 -- 172.21.15.50:0/638544155 learned_addr learned my addr 172.21.15.50:0/638544155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f920fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47634/0 (socket says 172.21.15.50:47634) 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9215800640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 -- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 msgr2=0x558c04c4c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c4c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 -- 172.21.15.50:0/638544155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 msgr2=0x558c04c452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f9214e00640 1 -- 172.21.15.50:0/638544155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0497d620 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.108+0000 7f920fe00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.109+0000 7f9214e00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f920801b040 tx=0x7f9208004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6cb62516a3f45b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.109+0000 7f91fea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.109+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208027020 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.109+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208003e20 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.109+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9208003ab0 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f920801d040 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 10 client.25744 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c04c4f180 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9208003e20 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9208020030 con 0x558c048702f0 2024-05-10T21:10:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 10 client.25744 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 20 client.25744 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c04857d80 0x558c04c52670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 4 client.25744 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.110+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c04bad160 con 0x558c04857d80 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.111+0000 7f920fe00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c04857d80 0x558c04c52670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.111+0000 7f920fe00640 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c04857d80 0x558c04c52670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3794ca5735f9e4 server_cookie=c2d8d6ed48afd3b in_seq=0 out_seq=0 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.111+0000 7f91fea00640 10 client.25744 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f91fea00640 1 -- 172.21.15.50:0/638544155 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c04bad160 con 0x558c04857d80 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f91fea00640 10 client.25744 handle_command_reply: tid=0 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f921e856740 1 client.25744 shutdown 2024-05-10T21:10:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 msgr2=0x558c04c44d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f920801b040 tx=0x7f9208004840 comp rx=0 tx=0).stop 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 shutdown_connections 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.112+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c04857d80 0x558c04c52670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c4c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f91fea00640 0 client.25744 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 --2- 172.21.15.50:0/638544155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c44d60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 >> 172.21.15.50:0/638544155 conn(0x558c0488d300 msgr2=0x558c04c39ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 shutdown_connections 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 1 -- 172.21.15.50:0/638544155 wait complete. 2024-05-10T21:10:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.113+0000 7f921e856740 20 client.25744 trim_cache size 0 max 16384 2024-05-10T21:10:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 Processor -- start 2024-05-10T21:10:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 -- start start 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048702f0 0x558c04c43d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048991b0 0x558c04c4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92101afbe0 con 0x558c048991b0 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ad4b0 con 0x558c048702f0 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.130+0000 7f921e856740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ab300 con 0x558c048784d0 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58808/0 (socket says 172.21.15.50:58808) 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f9215800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048991b0 0x558c04c4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 -- 172.21.15.50:0/850523197 learned_addr learned my addr 172.21.15.50:0/850523197 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f9214e00640 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048702f0 0x558c04c43d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 -- 172.21.15.50:0/850523197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048702f0 msgr2=0x558c04c43d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048702f0 0x558c04c43d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 -- 172.21.15.50:0/850523197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048991b0 msgr2=0x558c04c4c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048991b0 0x558c04c4c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.131+0000 7f920fe00640 1 -- 172.21.15.50:0/850523197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c04c430b0 con 0x558c048784d0 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.132+0000 7f920fe00640 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91f80a9900 tx=0x7f91f80e7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e445cb4b3337e0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.132+0000 7f9205600640 1 -- 172.21.15.50:0/850523197 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8091040 con 0x558c048784d0 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.132+0000 7f9205600640 1 -- 172.21.15.50:0/850523197 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f8010720 con 0x558c048784d0 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.132+0000 7f9205600640 1 -- 172.21.15.50:0/850523197 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b29b0 con 0x558c048784d0 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- 172.21.15.50:0/850523197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 msgr2=0x558c04c45040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91f80a9900 tx=0x7f91f80e7040 comp rx=0 tx=0).stop 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- 172.21.15.50:0/850523197 shutdown_connections 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048784d0 0x558c04c45040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048991b0 0x558c04c4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 --2- 172.21.15.50:0/850523197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048702f0 0x558c04c43d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- 172.21.15.50:0/850523197 >> 172.21.15.50:0/850523197 conn(0x558c0488d300 msgr2=0x558c04c44930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- 172.21.15.50:0/850523197 shutdown_connections 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- 172.21.15.50:0/850523197 wait complete. 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.133+0000 7f921e856740 1 -- start start 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 Processor -- start 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c4c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c48db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ab970 con 0x558c048784d0 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f92101acde0 con 0x558c048991b0 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ac040 con 0x558c048702f0 2024-05-10T21:10:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f921e856740 10 client.0 _ll_register_callbacks cb 0x7f92080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f920fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c4c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9215800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c48db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.134+0000 7f920fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47652/0 (socket says 172.21.15.50:47652) 2024-05-10T21:10:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f920fe00640 1 -- 172.21.15.50:0/3324153079 learned_addr learned my addr 172.21.15.50:0/3324153079 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 -- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 msgr2=0x558c04c48db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c48db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 -- 172.21.15.50:0/3324153079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 msgr2=0x558c04c4c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 -- 172.21.15.50:0/3324153079 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c04c49420 con 0x558c048702f0 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f920fe00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f9214e00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c4c230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920800fe30 tx=0x7f9208009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b0236dd30dead5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.135+0000 7f91fea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:10:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208027040 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f920800f7a0 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c04ba6fd0 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f920803d070 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f921e856740 10 client.25750 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c04cb95a0 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.136+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558c04ba6fd0 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9208035030 con 0x558c048702f0 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 10 client.25750 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 20 client.25750 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c0482d1c0 0x558c04cbbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 4 client.25750 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:10:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c04b224b0 con 0x558c0482d1c0 2024-05-10T21:10:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.137+0000 7f920fe00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c0482d1c0 0x558c04cbbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.138+0000 7f920fe00640 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c0482d1c0 0x558c04cbbe70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4987e1ccc9d43ca2 server_cookie=f610f5204e5a4b6f in_seq=0 out_seq=0 2024-05-10T21:10:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.138+0000 7f91fea00640 10 client.25750 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f91fea00640 1 -- 172.21.15.50:0/3324153079 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c04b224b0 con 0x558c0482d1c0 2024-05-10T21:10:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f91fea00640 10 client.25750 handle_command_reply: tid=0 2024-05-10T21:10:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 client.25750 shutdown 2024-05-10T21:10:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 msgr2=0x558c04c4c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c4c230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f920800fe30 tx=0x7f9208009ea0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 shutdown_connections 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x558c0482d1c0 0x558c04cbbe70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c048991b0 0x558c04c48db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c048784d0 0x558c04c4c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 --2- 172.21.15.50:0/3324153079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558c048702f0 0x558c04c4c230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 >> 172.21.15.50:0/3324153079 conn(0x558c0488d300 msgr2=0x558c04c3a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f91fea00640 0 client.25750 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 shutdown_connections 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.142+0000 7f921e856740 1 -- 172.21.15.50:0/3324153079 wait complete. 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f921e856740 20 client.25750 trim_cache size 0 max 16384 2024-05-10T21:10:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:10:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:10:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:10:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:10:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 msgr2=0x7f91f40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 0x7f91f40b4cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9200031440 tx=0x7f92000314a0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 msgr2=0x7f921019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91f800e9f0 tx=0x7f91f800eeb0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.143+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 shutdown_connections 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91f40b27e0 0x7f91f40b4cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9210109ad0 0x7f92101a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9210104dd0 0x7f921019c8d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 --2- 172.21.15.50:0/1278534141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9210103b90 0x7f921019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 >> 172.21.15.50:0/1278534141 conn(0x7f92100fed80 msgr2=0x7f92100ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 shutdown_connections 2024-05-10T21:10:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:10:47.144+0000 7f9216200640 1 -- 172.21.15.50:0/1278534141 wait complete. 2024-05-10T21:10:47.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:46 smithi050 ceph-mon[29546]: pgmap v517: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:47.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:46 smithi092 ceph-mon[34415]: pgmap v517: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:46 smithi142 ceph-mon[34846]: pgmap v517: 97 pgs: 97 active+clean; 675 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.742+0000 7fa3fde00700 1 -- 172.21.15.50:0/3250610202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8110ba0 msgr2=0x7fa3f8113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.742+0000 7fa3fde00700 1 --2- 172.21.15.50:0/3250610202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8110ba0 0x7fa3f8113000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009f90 tx=0x7fa3f000b330 comp rx=0 tx=0).stop 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 -- 172.21.15.50:0/3250610202 shutdown_connections 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 --2- 172.21.15.50:0/3250610202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8110ba0 0x7fa3f8113000 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 --2- 172.21.15.50:0/3250610202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8075d30 0x7fa3f8076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 --2- 172.21.15.50:0/3250610202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 0x7fa3f8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 -- 172.21.15.50:0/3250610202 >> 172.21.15.50:0/3250610202 conn(0x7fa3f80fffe0 msgr2=0x7fa3f8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 -- 172.21.15.50:0/3250610202 shutdown_connections 2024-05-10T21:10:48.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.743+0000 7fa3fde00700 1 -- 172.21.15.50:0/3250610202 wait complete. 2024-05-10T21:10:48.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.744+0000 7fa3fde00700 1 Processor -- start 2024-05-10T21:10:48.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.744+0000 7fa3fde00700 1 -- start start 2024-05-10T21:10:48.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 0x7fa3f819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8110ba0 0x7fa3f81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81166d0 con 0x7fa3f8110ba0 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f8116550 con 0x7fa3f8076680 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3fde00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f8116850 con 0x7fa3f8075d30 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58844/0 (socket says 172.21.15.50:58844) 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6a00700 1 -- 172.21.15.50:0/2030248608 learned_addr learned my addr 172.21.15.50:0/2030248608 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6000700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 0x7fa3f819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:48.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f7400700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8110ba0 0x7fa3f81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6a00700 1 -- 172.21.15.50:0/2030248608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 msgr2=0x7fa3f819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.745+0000 7fa3f6a00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 0x7fa3f819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3f6a00700 1 -- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8110ba0 msgr2=0x7fa3f81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3f6a00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8110ba0 0x7fa3f81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3f6a00700 1 -- 172.21.15.50:0/2030248608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f81a1ef0 con 0x7fa3f8075d30 2024-05-10T21:10:48.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3f6a00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9c0 tx=0x7fa3e800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004a30 con 0x7fa3f8075d30 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e800cc70 con 0x7fa3f8075d30 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8005490 con 0x7fa3f8075d30 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f81a21e0 con 0x7fa3f8075d30 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.746+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f81a9b00 con 0x7fa3f8075d30 2024-05-10T21:10:48.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.748+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f80512c0 con 0x7fa3f8075d30 2024-05-10T21:10:48.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.750+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3e8004bd0 con 0x7fa3f8075d30 2024-05-10T21:10:48.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.752+0000 7fa3fca00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:48.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.752+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fa3e80e2250 con 0x7fa3f8075d30 2024-05-10T21:10:48.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.752+0000 7fa3f6000700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:48.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.753+0000 7fa3f6000700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8103450 tx=0x7fa3e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:48.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:48.754+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3e80a9f40 con 0x7fa3f8075d30 2024-05-10T21:10:49.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:48 smithi050 ceph-mon[29546]: pgmap v518: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:10:49.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:48 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:10:49.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.047+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f802e5f0 con 0x7fa3f8075d30 2024-05-10T21:10:49.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.049+0000 7fa3fca00700 1 -- 172.21.15.50:0/2030248608 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fa3e80a9840 con 0x7fa3f8075d30 2024-05-10T21:10:49.048 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:10:49.049 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:10:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 msgr2=0x7fa3dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8103450 tx=0x7fa3e4009260 comp rx=0 tx=0).stop 2024-05-10T21:10:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 msgr2=0x7fa3f819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9c0 tx=0x7fa3e800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:10:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 shutdown_connections 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa3dc0b2900 0x7fa3dc0b4e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3f8110ba0 0x7fa3f81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8075d30 0x7fa3f819d020 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 --2- 172.21.15.50:0/2030248608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f8076680 0x7fa3f819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.052+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 >> 172.21.15.50:0/2030248608 conn(0x7fa3f80fffe0 msgr2=0x7fa3f8101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.053+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 shutdown_connections 2024-05-10T21:10:49.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:49.053+0000 7fa3fde00700 1 -- 172.21.15.50:0/2030248608 wait complete. 2024-05-10T21:10:49.053 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:10:49.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:48 smithi092 ceph-mon[34415]: pgmap v518: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:10:49.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:48 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:10:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:48 smithi142 ceph-mon[34846]: pgmap v518: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:10:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:48 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:10:49.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:10:50.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:49 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2030248608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:50.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:10:50.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:10:50.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:10:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:50.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:49 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2030248608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:50.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:10:50.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:10:50.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:10:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:49 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2030248608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:10:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:10:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:10:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:10:51.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:50 smithi050 ceph-mon[29546]: pgmap v519: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:51.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:50 smithi092 ceph-mon[34415]: pgmap v519: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:50 smithi142 ceph-mon[34846]: pgmap v519: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:52.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:52 smithi050 ceph-mon[29546]: pgmap v520: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:52.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:52 smithi092 ceph-mon[34415]: pgmap v520: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:52.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:52 smithi142 ceph-mon[34846]: pgmap v520: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 -- 172.21.15.50:0/3962679063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 msgr2=0x7fd174110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 --2- 172.21.15.50:0/3962679063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd174110db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b0a0 tx=0x7fd16c00b3b0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 -- 172.21.15.50:0/3962679063 shutdown_connections 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 --2- 172.21.15.50:0/3962679063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd174105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 --2- 172.21.15.50:0/3962679063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd174110db0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 --2- 172.21.15.50:0/3962679063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 0x7fd17410e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 -- 172.21.15.50:0/3962679063 >> 172.21.15.50:0/3962679063 conn(0x7fd1740fedc0 msgr2=0x7fd1741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 -- 172.21.15.50:0/3962679063 shutdown_connections 2024-05-10T21:10:52.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.646+0000 7fd17ac00700 1 -- 172.21.15.50:0/3962679063 wait complete. 2024-05-10T21:10:52.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.647+0000 7fd17ac00700 1 Processor -- start 2024-05-10T21:10:52.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.647+0000 7fd17ac00700 1 -- start start 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd17419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 0x7fd17419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd174114480 con 0x7fd17410e8d0 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd174114300 con 0x7fd174105460 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd17ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd174114600 con 0x7fd174105db0 2024-05-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd178c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd178c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47706/0 (socket says 172.21.15.50:47706) 2024-05-10T21:10:52.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd178c00700 1 -- 172.21.15.50:0/1379579423 learned_addr learned my addr 172.21.15.50:0/1379579423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:52.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd173e00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd17419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.648+0000 7fd173400700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 0x7fd17419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 -- 172.21.15.50:0/1379579423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 msgr2=0x7fd17419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 0x7fd17419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 -- 172.21.15.50:0/1379579423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 msgr2=0x7fd17419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd17419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 -- 172.21.15.50:0/1379579423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1741a1f00 con 0x7fd17410e8d0 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd173e00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd17419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:52.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd178c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b070 tx=0x7fd16c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.649+0000 7fd170c00700 1 -- 172.21.15.50:0/1379579423 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c021070 con 0x7fd17410e8d0 2024-05-10T21:10:52.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.650+0000 7fd170c00700 1 -- 172.21.15.50:0/1379579423 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16c004e40 con 0x7fd17410e8d0 2024-05-10T21:10:52.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.650+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1741a2190 con 0x7fd17410e8d0 2024-05-10T21:10:52.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.650+0000 7fd170c00700 1 -- 172.21.15.50:0/1379579423 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c004630 con 0x7fd17410e8d0 2024-05-10T21:10:52.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.650+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1741a9b10 con 0x7fd17410e8d0 2024-05-10T21:10:52.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.654+0000 7fd170c00700 1 -- 172.21.15.50:0/1379579423 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd16c007980 con 0x7fd17410e8d0 2024-05-10T21:10:52.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.654+0000 7fd18975fb80 1 Processor -- start 2024-05-10T21:10:52.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 -- start start 2024-05-10T21:10:52.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d5131532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d51314d9f0 0x55d51314de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d51306fa20 con 0x55d512fcfe40 2024-05-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d51300d760 con 0x55d513153830 2024-05-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd18975fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d51306b9e0 con 0x55d51314d9f0 2024-05-10T21:10:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.655+0000 7fd170c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 0x7fd1580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49162/0 (socket says 172.21.15.50:49162) 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd170c00700 1 -- 172.21.15.50:0/1379579423 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fd16c00f020 con 0x7fd17410e8d0 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 -- 172.21.15.50:0/1885188159 learned_addr learned my addr 172.21.15.50:0/1885188159 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173400700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d5131532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd178c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 0x7fd1580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173400700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d51314d9f0 0x55d51314de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 -- 172.21.15.50:0/1885188159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d51314d9f0 msgr2=0x55d51314de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d51314d9f0 0x55d51314de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 -- 172.21.15.50:0/1885188159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 msgr2=0x55d5131532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d5131532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173e00700 1 -- 172.21.15.50:0/1885188159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d512fbc8d0 con 0x55d513153830 2024-05-10T21:10:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.656+0000 7fd173400700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d5131532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:10:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.657+0000 7fd173e00700 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd1640115f0 tx=0x7fd164011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4375b946be03c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.657+0000 7fd178c00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 0x7fd1580b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009bc0 tx=0x7fd16c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.657+0000 7fd169600700 1 -- 172.21.15.50:0/1885188159 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16401e070 con 0x55d513153830 2024-05-10T21:10:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.657+0000 7fd169600700 1 -- 172.21.15.50:0/1885188159 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16400fb30 con 0x55d513153830 2024-05-10T21:10:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.657+0000 7fd169600700 1 -- 172.21.15.50:0/1885188159 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164019bc0 con 0x55d513153830 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 -- 172.21.15.50:0/1885188159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 msgr2=0x55d51314d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd1640115f0 tx=0x7fd164011900 comp rx=0 tx=0).stop 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 -- 172.21.15.50:0/1885188159 shutdown_connections 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d51314d9f0 0x55d51314de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d5131532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1885188159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d51314d4b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 -- 172.21.15.50:0/1885188159 >> 172.21.15.50:0/1885188159 conn(0x55d513014f70 msgr2=0x55d5131482d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 -- 172.21.15.50:0/1885188159 shutdown_connections 2024-05-10T21:10:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.658+0000 7fd18975fb80 1 -- 172.21.15.50:0/1885188159 wait complete. 2024-05-10T21:10:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.659+0000 7fd18975fb80 1 -- start start 2024-05-10T21:10:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.659+0000 7fd18975fb80 1 Processor -- start 2024-05-10T21:10:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d5131574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d513157a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d51315a310 con 0x55d512fcfe40 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d512fd5b20 con 0x55d51314d9f0 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d51315a490 con 0x55d513153830 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd18975fb80 10 client.0 _ll_register_callbacks cb 0x55d513016c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd173400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd173400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47712/0 (socket says 172.21.15.50:47712) 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd173400700 1 -- 172.21.15.50:0/4177664645 learned_addr learned my addr 172.21.15.50:0/4177664645 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.660+0000 7fd173e00700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d5131574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d513157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 -- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 msgr2=0x55d513157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d513157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 -- 172.21.15.50:0/4177664645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 msgr2=0x55d5131574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d5131574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 -- 172.21.15.50:0/4177664645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d513035420 con 0x55d512fcfe40 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173e00700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d5131574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd173400700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd160012b90 tx=0x7fd1600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c6b785075dd739 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd15e000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.661+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004260 con 0x55d512fcfe40 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fd160004400 con 0x55d512fcfe40 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16400fb30 con 0x55d512fcfe40 2024-05-10T21:10:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd16000f910 con 0x55d512fcfe40 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd18975fb80 10 client.15669 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d513164ec0 con 0x55d512fcfe40 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16400fb30 con 0x55d512fcfe40 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.662+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd16001f020 con 0x55d512fcfe40 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 10 client.15669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 20 client.15669 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d513167500 0x55d513169920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 4 client.15669 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d51311a1c0 con 0x55d513167500 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.663+0000 7fd173e00700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d513167500 0x55d513169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.664+0000 7fd173e00700 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d513167500 0x55d513169920 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a482ecca8d0db48 server_cookie=af0ff346c41c6849 in_seq=0 out_seq=0 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.664+0000 7fd15e000700 10 client.15669 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd15e000700 1 -- 172.21.15.50:0/4177664645 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d51311a1c0 con 0x55d513167500 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd15e000700 10 client.15669 handle_command_reply: tid=0 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 client.15669 shutdown 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 msgr2=0x55d513156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd160012b90 tx=0x7fd1600047c0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 shutdown_connections 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d513167500 0x55d513169920 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d513157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d513156f80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 --2- 172.21.15.50:0/4177664645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d5131574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 >> 172.21.15.50:0/4177664645 conn(0x55d513014f70 msgr2=0x55d51314c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd15e000700 0 client.15669 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.665+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 shutdown_connections 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.666+0000 7fd18975fb80 1 -- 172.21.15.50:0/4177664645 wait complete. 2024-05-10T21:10:52.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.666+0000 7fd18975fb80 20 client.15669 trim_cache size 0 max 16384 2024-05-10T21:10:52.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 Processor -- start 2024-05-10T21:10:52.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 -- start start 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d512fcfe40 0x55d51314d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d513157840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741b3500 con 0x55d51314d9f0 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741b0d90 con 0x55d513153830 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.683+0000 7fd18975fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741aec20 con 0x55d512fcfe40 2024-05-10T21:10:52.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d512fcfe40 0x55d51314d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47722/0 (socket says 172.21.15.50:47722) 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 -- 172.21.15.50:0/183830668 learned_addr learned my addr 172.21.15.50:0/183830668 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173400700 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d513157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 -- 172.21.15.50:0/183830668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d512fcfe40 msgr2=0x55d51314d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d512fcfe40 0x55d51314d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 -- 172.21.15.50:0/183830668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 msgr2=0x55d513157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d513157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 -- 172.21.15.50:0/183830668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d513149b50 con 0x55d51314d9f0 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.684+0000 7fd173e00700 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821c93363b99809d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.685+0000 7fd168c00700 1 -- 172.21.15.50:0/183830668 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16402c020 con 0x55d51314d9f0 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.685+0000 7fd168c00700 1 -- 172.21.15.50:0/183830668 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16401f040 con 0x55d51314d9f0 2024-05-10T21:10:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.685+0000 7fd168c00700 1 -- 172.21.15.50:0/183830668 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16401e040 con 0x55d51314d9f0 2024-05-10T21:10:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- 172.21.15.50:0/183830668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 msgr2=0x55d513148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640058e0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- 172.21.15.50:0/183830668 shutdown_connections 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d513153830 0x55d513157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d512fcfe40 0x55d51314d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 --2- 172.21.15.50:0/183830668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d51314d9f0 0x55d513148340 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- 172.21.15.50:0/183830668 >> 172.21.15.50:0/183830668 conn(0x55d513014f70 msgr2=0x55d513150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- 172.21.15.50:0/183830668 shutdown_connections 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- 172.21.15.50:0/183830668 wait complete. 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.686+0000 7fd18975fb80 1 -- start start 2024-05-10T21:10:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 Processor -- start 2024-05-10T21:10:52.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d51315f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d51316e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d51316e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741af270 con 0x55d512fcfe40 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741b06c0 con 0x55d51314d9f0 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741af920 con 0x55d513153830 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.687+0000 7fd18975fb80 10 client.0 _ll_register_callbacks cb 0x55d513016c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d51315f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d51315f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47734/0 (socket says 172.21.15.50:47734) 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173400700 1 -- 172.21.15.50:0/1467273230 learned_addr learned my addr 172.21.15.50:0/1467273230 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d51316e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173400700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d51316e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 -- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 msgr2=0x55d51316e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d51316e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 -- 172.21.15.50:0/1467273230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 msgr2=0x55d51315f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d51315f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 -- 172.21.15.50:0/1467273230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d513160040 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd173e00700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d51316e0e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55d513159c20 tx=0x7fd164005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54a9cba95c7848f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.688+0000 7fd15e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16402c070 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fd16400fa50 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16401f040 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd16401e040 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd18975fb80 10 client.25802 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.689+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5131d9e10 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16401f040 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd1640052c0 con 0x55d51314d9f0 2024-05-10T21:10:52.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 10 client.25802 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:52.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 10 client.25802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:10:52.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 20 client.25802 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:52.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d5131da260 0x55d5131dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:52.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 4 client.25802 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:52.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d51307d400 con 0x55d5131da260 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.690+0000 7fd173400700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d5131da260 0x55d5131dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd173400700 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d5131da260 0x55d5131dc680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=609b879583585d0e server_cookie=6adfca3ce92f6281 in_seq=0 out_seq=0 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd15e000700 10 client.25802 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd15e000700 1 -- 172.21.15.50:0/1467273230 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d51307d400 con 0x55d5131da260 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd15e000700 10 client.25802 handle_command_reply: tid=0 2024-05-10T21:10:52.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 client.25802 shutdown 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 msgr2=0x55d51316e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d51316e0e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55d513159c20 tx=0x7fd164005fa0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 shutdown_connections 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55d5131da260 0x55d5131dc680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d513153830 0x55d51316e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d512fcfe40 0x55d51315f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 --2- 172.21.15.50:0/1467273230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d51314d9f0 0x55d51316e0e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 >> 172.21.15.50:0/1467273230 conn(0x55d513014f70 msgr2=0x55d513152870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.691+0000 7fd15e000700 0 client.25802 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 shutdown_connections 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd18975fb80 1 -- 172.21.15.50:0/1467273230 wait complete. 2024-05-10T21:10:52.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd18975fb80 20 client.25802 trim_cache size 0 max 16384 2024-05-10T21:10:52.694 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 msgr2=0x7fd1580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 0x7fd1580b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009bc0 tx=0x7fd16c009040 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 msgr2=0x7fd1741a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.692+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b070 tx=0x7fd16c009c90 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 shutdown_connections 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1580b2900 0x7fd1580b4e10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd174105460 0x7fd17419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd17410e8d0 0x7fd1741a1860 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 --2- 172.21.15.50:0/1379579423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd174105db0 0x7fd17419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 >> 172.21.15.50:0/1379579423 conn(0x7fd1740fedc0 msgr2=0x7fd1741031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 shutdown_connections 2024-05-10T21:10:52.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:52.693+0000 7fd17ac00700 1 -- 172.21.15.50:0/1379579423 wait complete. 2024-05-10T21:10:53.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:10:54.124 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:54.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:54 smithi050 ceph-mon[29546]: pgmap v521: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:10:55.071 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:54 smithi092 ceph-mon[34415]: pgmap v521: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:10:55.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:54 smithi142 ceph-mon[34846]: pgmap v521: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.478+0000 7f961d600700 1 -- 172.21.15.50:0/3727353972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618106580 msgr2=0x7f9618106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.478+0000 7f961d600700 1 --2- 172.21.15.50:0/3727353972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618106580 0x7f9618106990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96000098f0 tx=0x7f9600009c00 comp rx=0 tx=0).stop 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.478+0000 7f961d600700 1 -- 172.21.15.50:0/3727353972 shutdown_connections 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 --2- 172.21.15.50:0/3727353972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106ed0 0x7f9618113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 --2- 172.21.15.50:0/3727353972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f961810c5c0 0x7f961810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 --2- 172.21.15.50:0/3727353972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618106580 0x7f9618106990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 -- 172.21.15.50:0/3727353972 >> 172.21.15.50:0/3727353972 conn(0x7f9618102080 msgr2=0x7f96181044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 -- 172.21.15.50:0/3727353972 shutdown_connections 2024-05-10T21:10:56.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.479+0000 7f961d600700 1 -- 172.21.15.50:0/3727353972 wait complete. 2024-05-10T21:10:56.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.480+0000 7f961d600700 1 Processor -- start 2024-05-10T21:10:56.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.480+0000 7f961d600700 1 -- start start 2024-05-10T21:10:56.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9618106ed0 0x7f961819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961810c5c0 0x7f96181a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96181166a0 con 0x7f9618106ed0 2024-05-10T21:10:56.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9618116520 con 0x7f961810c5c0 2024-05-10T21:10:56.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f961d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9618116820 con 0x7f9618106580 2024-05-10T21:10:56.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f9616a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f960fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9618106ed0 0x7f961819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f9616a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43730/0 (socket says 172.21.15.50:43730) 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f9616a00700 1 -- 172.21.15.50:0/3844737231 learned_addr learned my addr 172.21.15.50:0/3844737231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.481+0000 7f9617400700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961810c5c0 0x7f96181a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 -- 172.21.15.50:0/3844737231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961810c5c0 msgr2=0x7f96181a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961810c5c0 0x7f96181a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 -- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9618106ed0 msgr2=0x7f961819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9618106ed0 0x7f961819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 -- 172.21.15.50:0/3844737231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96181a1fe0 con 0x7f9618106580 2024-05-10T21:10:56.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f9616a00700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f960800e970 tx=0x7f960800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.482+0000 7f960f400700 1 -- 172.21.15.50:0/3844737231 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608004a30 con 0x7f9618106580 2024-05-10T21:10:56.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.483+0000 7f960f400700 1 -- 172.21.15.50:0/3844737231 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960800cc90 con 0x7f9618106580 2024-05-10T21:10:56.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.483+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181a22d0 con 0x7f9618106580 2024-05-10T21:10:56.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.483+0000 7f960f400700 1 -- 172.21.15.50:0/3844737231 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608005610 con 0x7f9618106580 2024-05-10T21:10:56.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.483+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181a9bf0 con 0x7f9618106580 2024-05-10T21:10:56.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.487+0000 7f960f400700 1 -- 172.21.15.50:0/3844737231 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f960800c7e0 con 0x7f9618106580 2024-05-10T21:10:56.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 Processor -- start 2024-05-10T21:10:56.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 -- start start 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f960f400700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 0x7f95fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc67e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6706a20 con 0x55bcc67e49f0 2024-05-10T21:10:56.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bcc66a4760 con 0x55bcc67ea830 2024-05-10T21:10:56.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f962c1dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc67029e0 con 0x55bcc6666e40 2024-05-10T21:10:56.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.488+0000 7f960f400700 1 -- 172.21.15.50:0/3844737231 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f96080e2330 con 0x7f9618106580 2024-05-10T21:10:56.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43734/0 (socket says 172.21.15.50:43734) 2024-05-10T21:10:56.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 -- 172.21.15.50:0/1669291273 learned_addr learned my addr 172.21.15.50:0/1669291273 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9617400700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 0x7f95fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f960fe00700 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9617400700 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc67e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 -- 172.21.15.50:0/1669291273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 msgr2=0x55bcc67e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 -- 172.21.15.50:0/1669291273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 msgr2=0x55bcc67e4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc67e4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.489+0000 7f9616a00700 1 -- 172.21.15.50:0/1669291273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc66538d0 con 0x55bcc6666e40 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.490+0000 7f9616a00700 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9608009230 tx=0x7f96080050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d828c6e15cfc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.490+0000 7f9607e00700 1 -- 172.21.15.50:0/1669291273 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080a9090 con 0x55bcc6666e40 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.490+0000 7f9607e00700 1 -- 172.21.15.50:0/1669291273 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960800cad0 con 0x55bcc6666e40 2024-05-10T21:10:56.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.490+0000 7f9607e00700 1 -- 172.21.15.50:0/1669291273 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080a50b0 con 0x55bcc6666e40 2024-05-10T21:10:56.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 -- 172.21.15.50:0/1669291273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 msgr2=0x55bcc67ea2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9608009230 tx=0x7f96080050b0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f9617400700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 0x7f95fc0b4d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9610008920 tx=0x7f96100099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 -- 172.21.15.50:0/1669291273 shutdown_connections 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc67e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc6666e40 0x55bcc67ea2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1669291273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.491+0000 7f962c1dab80 1 -- 172.21.15.50:0/1669291273 >> 172.21.15.50:0/1669291273 conn(0x55bcc66abf70 msgr2=0x55bcc67df2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.492+0000 7f962c1dab80 1 -- 172.21.15.50:0/1669291273 shutdown_connections 2024-05-10T21:10:56.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.492+0000 7f962c1dab80 1 -- 172.21.15.50:0/1669291273 wait complete. 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.492+0000 7f962c1dab80 1 -- start start 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.492+0000 7f962c1dab80 1 Processor -- start 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67e49f0 0x55bcc67ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bcc67f1310 con 0x55bcc6666e40 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bcc666cb20 con 0x55bcc67ea830 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc67f1490 con 0x55bcc67e49f0 2024-05-10T21:10:56.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f962c1dab80 10 client.0 _ll_register_callbacks cb 0x55bcc66adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:56.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.493+0000 7f9616a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f960fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67e49f0 0x55bcc67ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9616a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60566/0 (socket says 172.21.15.50:60566) 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54980/0 (socket says 172.21.15.50:54980) 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9616a00700 1 -- 172.21.15.50:0/1372006342 learned_addr learned my addr 172.21.15.50:0/1372006342 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 -- 172.21.15.50:0/1372006342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67e49f0 msgr2=0x55bcc67ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67e49f0 0x55bcc67ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 -- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 msgr2=0x55bcc67ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 -- 172.21.15.50:0/1372006342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc66cc420 con 0x55bcc67ea830 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9616a00700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9617400700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9610018640 tx=0x7f9610004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1404738812295a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.494+0000 7f9604c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961002a060 con 0x55bcc67ea830 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcc67b11c0 con 0x55bcc67ea830 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f9610018d90 con 0x55bcc67ea830 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96100075c0 con 0x55bcc67ea830 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f962c1dab80 10 client.25814 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:56.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.495+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcc67fbf50 con 0x55bcc67ea830 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bcc67b11c0 con 0x55bcc67ea830 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f961001a8c0 con 0x55bcc67ea830 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 10 client.25814 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 10 client.25814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 20 client.25814 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc67fe4e0 0x55bcc6800900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 4 client.25814 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:10:56.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc6714400 con 0x55bcc67fe4e0 2024-05-10T21:10:56.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.496+0000 7f9616a00700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc67fe4e0 0x55bcc6800900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.497+0000 7f9616a00700 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc67fe4e0 0x55bcc6800900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e42793fc342e7702 server_cookie=7b57adf98562caa in_seq=0 out_seq=0 2024-05-10T21:10:56.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.497+0000 7f9604c00700 10 client.25814 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f9604c00700 1 -- 172.21.15.50:0/1372006342 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcc6714400 con 0x55bcc67fe4e0 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f9604c00700 10 client.25814 handle_command_reply: tid=0 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 client.25814 shutdown 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 msgr2=0x55bcc67eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9610018640 tx=0x7f9610004b40 comp rx=0 tx=0).stop 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 shutdown_connections 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67ea830 0x55bcc67eeac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc67fe4e0 0x55bcc6800900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 --2- 172.21.15.50:0/1372006342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67e49f0 0x55bcc67ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f9604c00700 0 client.25814 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:56.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.498+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 >> 172.21.15.50:0/1372006342 conn(0x55bcc66abf70 msgr2=0x55bcc67e34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.499+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 shutdown_connections 2024-05-10T21:10:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.499+0000 7f962c1dab80 1 -- 172.21.15.50:0/1372006342 wait complete. 2024-05-10T21:10:56.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.499+0000 7f962c1dab80 20 client.25814 trim_cache size 0 max 16384 2024-05-10T21:10:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 Processor -- start 2024-05-10T21:10:56.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 -- start start 2024-05-10T21:10:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 0x55bcc67f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc67ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b3760 con 0x55bcc6666e40 2024-05-10T21:10:56.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b0ff0 con 0x55bcc67e49f0 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.516+0000 7f962c1dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aee60 con 0x55bcc67ea830 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f9616a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f9616a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60570/0 (socket says 172.21.15.50:60570) 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 0x55bcc67f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f9616a00700 1 -- 172.21.15.50:0/3364907206 learned_addr learned my addr 172.21.15.50:0/3364907206 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:56.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f9617400700 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc67ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 -- 172.21.15.50:0/3364907206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 msgr2=0x55bcc67ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc67ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 -- 172.21.15.50:0/3364907206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 msgr2=0x55bcc67e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.517+0000 7f960fe00700 1 -- 172.21.15.50:0/3364907206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc67ec5b0 con 0x55bcc67e49f0 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.518+0000 7f960fe00700 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 0x55bcc67f04e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aedaa8466f7825 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.518+0000 7f9607400700 1 -- 172.21.15.50:0/3364907206 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600005900 con 0x55bcc67e49f0 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.518+0000 7f9607400700 1 -- 172.21.15.50:0/3364907206 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960000f910 con 0x55bcc67e49f0 2024-05-10T21:10:56.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.518+0000 7f9607400700 1 -- 172.21.15.50:0/3364907206 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96000038e0 con 0x55bcc67e49f0 2024-05-10T21:10:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 -- 172.21.15.50:0/3364907206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 msgr2=0x55bcc67f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 0x55bcc67f04e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005dc0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 -- 172.21.15.50:0/3364907206 shutdown_connections 2024-05-10T21:10:56.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc67ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc6666e40 0x55bcc67e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 --2- 172.21.15.50:0/3364907206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc67e49f0 0x55bcc67f04e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 -- 172.21.15.50:0/3364907206 >> 172.21.15.50:0/3364907206 conn(0x55bcc66abf70 msgr2=0x55bcc67ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 -- 172.21.15.50:0/3364907206 shutdown_connections 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.519+0000 7f962c1dab80 1 -- 172.21.15.50:0/3364907206 wait complete. 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 -- start start 2024-05-10T21:10:56.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 Processor -- start 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 0x55bcc6805900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc6806380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96181af4b0 con 0x55bcc67e49f0 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b0920 con 0x55bcc6666e40 2024-05-10T21:10:56.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.520+0000 7f962c1dab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96181afb80 con 0x55bcc67ea830 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f962c1dab80 10 client.0 _ll_register_callbacks cb 0x55bcc66adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f960fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 0x55bcc6805900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f960fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60576/0 (socket says 172.21.15.50:60576) 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f960fe00700 1 -- 172.21.15.50:0/4225269715 learned_addr learned my addr 172.21.15.50:0/4225269715 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9617400700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc6806380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 -- 172.21.15.50:0/4225269715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 msgr2=0x55bcc6806380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc6806380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 -- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 msgr2=0x55bcc6805e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:10:56.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f9616a00700 1 -- 172.21.15.50:0/4225269715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc6867c70 con 0x55bcc6666e40 2024-05-10T21:10:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.521+0000 7f960fe00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:10:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9616a00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 0x55bcc6805900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96080a9620 tx=0x7f96080a94b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f091b6de1b1a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:10:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9604c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:10:56.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960809de60 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f96080a5f10 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960000f910 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f960800ce40 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f962c1dab80 10 client.25820 fetch_fsmap learned FSMap version 25 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcc6868db0 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.522+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960000f910 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9608098ce0 con 0x55bcc6666e40 2024-05-10T21:10:56.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 10 client.25820 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 10 client.25820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 20 client.25820 populate_metadata read hostname 'smithi050' 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc6868fe0 0x55bcc686b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 4 client.25820 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcc670e4a0 con 0x55bcc6868fe0 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.523+0000 7f960fe00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc6868fe0 0x55bcc686b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.524+0000 7f960fe00700 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc6868fe0 0x55bcc686b400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e01537a7a1d13f server_cookie=5356d3e4058847fd in_seq=0 out_seq=0 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.524+0000 7f9604c00700 10 client.25820 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.524+0000 7f9604c00700 1 -- 172.21.15.50:0/4225269715 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcc670e4a0 con 0x55bcc6868fe0 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.524+0000 7f9604c00700 10 client.25820 handle_command_reply: tid=0 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.524+0000 7f962c1dab80 1 client.25820 shutdown 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 msgr2=0x55bcc6805900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 0x55bcc6805900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96080a9620 tx=0x7f96080a94b0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 shutdown_connections 2024-05-10T21:10:56.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bcc6666e40 0x55bcc6805900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55bcc6868fe0 0x55bcc686b400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc67ea830 0x55bcc6806380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 --2- 172.21.15.50:0/4225269715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcc67e49f0 0x55bcc6805e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 >> 172.21.15.50:0/4225269715 conn(0x55bcc66abf70 msgr2=0x55bcc67edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f9604c00700 0 client.25820 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 shutdown_connections 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 1 -- 172.21.15.50:0/4225269715 wait complete. 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.525+0000 7f962c1dab80 20 client.25820 trim_cache size 0 max 16384 2024-05-10T21:10:56.525 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 msgr2=0x7f95fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 0x7f95fc0b4d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9610008920 tx=0x7f96100099e0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 msgr2=0x7f961819d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f960800e970 tx=0x7f960800ec80 comp rx=0 tx=0).stop 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 shutdown_connections 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f95fc0b2880 0x7f95fc0b4d90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961810c5c0 0x7f96181a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9618106580 0x7f961819d110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 --2- 172.21.15.50:0/3844737231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9618106ed0 0x7f961819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:10:56.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 >> 172.21.15.50:0/3844737231 conn(0x7f9618102080 msgr2=0x7f9618104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:10:56.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 shutdown_connections 2024-05-10T21:10:56.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:10:56.526+0000 7f961d600700 1 -- 172.21.15.50:0/3844737231 wait complete. 2024-05-10T21:10:56.626 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:10:57.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:56 smithi092 ceph-mon[34415]: pgmap v522: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:10:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:56 smithi142 ceph-mon[34846]: pgmap v522: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:10:57.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:56 smithi050 ceph-mon[29546]: pgmap v522: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:10:57.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:10:57.616 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:58.964 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:10:59.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:10:58 smithi092 ceph-mon[34415]: pgmap v523: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:10:58 smithi142 ceph-mon[34846]: pgmap v523: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:10:59.132 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:10:58 smithi050 ceph-mon[29546]: pgmap v523: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 -- 172.21.15.50:0/1265376226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 msgr2=0x7fc95c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 --2- 172.21.15.50:0/1265376226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c10e240 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc95402a410 comp rx=0 tx=0).stop 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 -- 172.21.15.50:0/1265376226 shutdown_connections 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 --2- 172.21.15.50:0/1265376226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c10e240 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 --2- 172.21.15.50:0/1265376226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c107120 0x7fc95c107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 --2- 172.21.15.50:0/1265376226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c105ee0 0x7fc95c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 -- 172.21.15.50:0/1265376226 >> 172.21.15.50:0/1265376226 conn(0x7fc95c1010b0 msgr2=0x7fc95c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 -- 172.21.15.50:0/1265376226 shutdown_connections 2024-05-10T21:11:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.363+0000 7fc962c00640 1 -- 172.21.15.50:0/1265376226 wait complete. 2024-05-10T21:11:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.364+0000 7fc962c00640 1 Processor -- start 2024-05-10T21:11:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.364+0000 7fc962c00640 1 -- start start 2024-05-10T21:11:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 0x7fc95c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c1a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1119c0 con 0x7fc95c105ee0 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c111840 con 0x7fc95c10be20 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc962c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c111b40 con 0x7fc95c107120 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc95be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc960c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c1a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc95be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60600/0 (socket says 172.21.15.50:60600) 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.365+0000 7fc95be00640 1 -- 172.21.15.50:0/3555004414 learned_addr learned my addr 172.21.15.50:0/3555004414 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95b400640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 0x7fc95c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 -- 172.21.15.50:0/3555004414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 msgr2=0x7fc95c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 0x7fc95c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 -- 172.21.15.50:0/3555004414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 msgr2=0x7fc95c1a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c1a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 -- 172.21.15.50:0/3555004414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95c1a3ff0 con 0x7fc95c105ee0 2024-05-10T21:11:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc960c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c1a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95b400640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 0x7fc95c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc95be00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e9f0 tx=0x7fc94c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.366+0000 7fc958c00640 1 -- 172.21.15.50:0/3555004414 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00ccc0 con 0x7fc95c105ee0 2024-05-10T21:11:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.367+0000 7fc958c00640 1 -- 172.21.15.50:0/3555004414 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc94c0035f0 con 0x7fc95c105ee0 2024-05-10T21:11:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.367+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95c1a42e0 con 0x7fc95c105ee0 2024-05-10T21:11:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.367+0000 7fc958c00640 1 -- 172.21.15.50:0/3555004414 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00ce60 con 0x7fc95c105ee0 2024-05-10T21:11:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.367+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95c1a4880 con 0x7fc95c105ee0 2024-05-10T21:11:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.370+0000 7fc969d7a740 1 Processor -- start 2024-05-10T21:11:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 -- start start 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc958c00640 1 -- 172.21.15.50:0/3555004414 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc94c010690 con 0x7fc95c105ee0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d2c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 0x5646a9d33580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5646a9c90550 con 0x5646a99694d0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646a9c35bc0 con 0x5646a99612f0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.371+0000 7fc969d7a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646a9c8c510 con 0x5646a998a1b0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc95b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 0x5646a9d33580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc95be00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43786/0 (socket says 172.21.15.50:43786) 2024-05-10T21:11:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc95b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 0x5646a9d33580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60614/0 (socket says 172.21.15.50:60614) 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 -- 172.21.15.50:0/732264832 learned_addr learned my addr 172.21.15.50:0/732264832 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 -- 172.21.15.50:0/732264832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 msgr2=0x5646a9d2c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d2c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 -- 172.21.15.50:0/732264832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 msgr2=0x5646a9d33580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 0x5646a9d33580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 -- 172.21.15.50:0/732264832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a99a2210 con 0x5646a998a1b0 2024-05-10T21:11:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.372+0000 7fc960c00640 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc954009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dfff147a478535 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc951200640 1 -- 172.21.15.50:0/732264832 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954035030 con 0x5646a998a1b0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc951200640 1 -- 172.21.15.50:0/732264832 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc954040070 con 0x5646a998a1b0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc951200640 1 -- 172.21.15.50:0/732264832 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954031dd0 con 0x5646a998a1b0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc958c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 0x7fc93c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc958c00640 1 -- 172.21.15.50:0/3555004414 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc94c0e2520 con 0x7fc95c105ee0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.373+0000 7fc95be00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 0x7fc93c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 -- 172.21.15.50:0/732264832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 msgr2=0x5646a9d2c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc954009650 comp rx=0 tx=0).stop 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc95be00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 0x7fc93c0b4d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5646a9d2d4e0 tx=0x7fc94c0e8060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 -- 172.21.15.50:0/732264832 shutdown_connections 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99694d0 0x5646a9d33580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d2c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 --2- 172.21.15.50:0/732264832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d2c100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 -- 172.21.15.50:0/732264832 >> 172.21.15.50:0/732264832 conn(0x5646a997e300 msgr2=0x5646a9d27a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 -- 172.21.15.50:0/732264832 shutdown_connections 2024-05-10T21:11:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.374+0000 7fc969d7a740 1 -- 172.21.15.50:0/732264832 wait complete. 2024-05-10T21:11:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.375+0000 7fc969d7a740 1 -- start start 2024-05-10T21:11:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.375+0000 7fc969d7a740 1 Processor -- start 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d3e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5646a9d33f70 con 0x5646a998a1b0 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646a9c49660 con 0x5646a99612f0 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646a9d340f0 con 0x5646a99694d0 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc969d7a740 10 client.0 _ll_register_callbacks cb 0x5646a9c13400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc95b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc95b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43796/0 (socket says 172.21.15.50:43796) 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc95b400640 1 -- 172.21.15.50:0/1630796208 learned_addr learned my addr 172.21.15.50:0/1630796208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.376+0000 7fc960c00640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d3e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc960c00640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc95b400640 1 -- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 msgr2=0x5646a9d3e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc95b400640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d3e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc95b400640 1 -- 172.21.15.50:0/1630796208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 msgr2=0x5646a9d45840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc95b400640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d45840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.377+0000 7fc95b400640 1 -- 172.21.15.50:0/1630796208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a9a6e620 con 0x5646a99694d0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc95b400640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc9440042f0 tx=0x7fc944003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53badbc0527863a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc949600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440119a0 con 0x5646a99694d0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc954040070 con 0x5646a99694d0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc944011b40 con 0x5646a99694d0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc944022070 con 0x5646a99694d0 2024-05-10T21:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.378+0000 7fc969d7a740 10 client.25810 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646a9d48890 con 0x5646a99694d0 2024-05-10T21:11:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc954040070 con 0x5646a99694d0 2024-05-10T21:11:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc944010620 con 0x5646a99694d0 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 10 client.25810 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 10 client.25810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 20 client.25810 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a9948d80 0x5646a9d4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 4 client.25810 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.379+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a9c9e160 con 0x5646a9948d80 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.380+0000 7fc960c00640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a9948d80 0x5646a9d4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.380+0000 7fc960c00640 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a9948d80 0x5646a9d4bf40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edeb191a13ed496 server_cookie=f12d2592d14adf04 in_seq=0 out_seq=0 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.380+0000 7fc949600640 10 client.25810 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.381+0000 7fc949600640 1 -- 172.21.15.50:0/1630796208 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646a9c9e160 con 0x5646a9948d80 2024-05-10T21:11:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.381+0000 7fc949600640 10 client.25810 handle_command_reply: tid=0 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.381+0000 7fc969d7a740 1 client.25810 shutdown 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.381+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 msgr2=0x5646a9d3e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.381+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc9440042f0 tx=0x7fc944003f90 comp rx=0 tx=0).stop 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 shutdown_connections 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a9948d80 0x5646a9d4bf40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d3e8e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1630796208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d3e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc949600640 0 client.25810 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 >> 172.21.15.50:0/1630796208 conn(0x5646a997e300 msgr2=0x5646a9d2a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 shutdown_connections 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 1 -- 172.21.15.50:0/1630796208 wait complete. 2024-05-10T21:11:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.382+0000 7fc969d7a740 20 client.25810 trim_cache size 0 max 16384 2024-05-10T21:11:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.399+0000 7fc969d7a740 1 Processor -- start 2024-05-10T21:11:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.399+0000 7fc969d7a740 1 -- start start 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.399+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 0x5646a9d34bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.399+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 0x5646a9d45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d45ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc969d7a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1ab8d0 con 0x5646a99612f0 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc969d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1a91a0 con 0x5646a99694d0 2024-05-10T21:11:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc969d7a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1a6ff0 con 0x5646a998a1b0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc960c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 0x5646a9d34bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc960c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 0x5646a9d34bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60628/0 (socket says 172.21.15.50:60628) 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc960c00640 1 -- 172.21.15.50:0/3439847723 learned_addr learned my addr 172.21.15.50:0/3439847723 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc95b400640 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 0x5646a9d45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.400+0000 7fc960c00640 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d45ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 -- 172.21.15.50:0/3439847723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 msgr2=0x5646a9d45ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d45ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 -- 172.21.15.50:0/3439847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 msgr2=0x5646a9d34bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 0x5646a9d34bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 -- 172.21.15.50:0/3439847723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a9d23280 con 0x5646a99694d0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc95b400640 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 0x5646a9d45760 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e400 tx=0x7fc944003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f3bff21bb1b198 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc94be00640 1 -- 172.21.15.50:0/3439847723 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944010df0 con 0x5646a99694d0 2024-05-10T21:11:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc94be00640 1 -- 172.21.15.50:0/3439847723 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9440119a0 con 0x5646a99694d0 2024-05-10T21:11:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.401+0000 7fc94be00640 1 -- 172.21.15.50:0/3439847723 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944027c40 con 0x5646a99694d0 2024-05-10T21:11:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.402+0000 7fc969d7a740 1 -- 172.21.15.50:0/3439847723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 msgr2=0x5646a9d45760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 0x5646a9d45760 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e400 tx=0x7fc944003eb0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 -- 172.21.15.50:0/3439847723 shutdown_connections 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a998a1b0 0x5646a9d45ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99694d0 0x5646a9d45760 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 --2- 172.21.15.50:0/3439847723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a99612f0 0x5646a9d34bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 -- 172.21.15.50:0/3439847723 >> 172.21.15.50:0/3439847723 conn(0x5646a997e300 msgr2=0x5646a9d344c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 -- 172.21.15.50:0/3439847723 shutdown_connections 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 -- 172.21.15.50:0/3439847723 wait complete. 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.403+0000 7fc969d7a740 1 -- start start 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 Processor -- start 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d44920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1a7660 con 0x5646a998a1b0 2024-05-10T21:11:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1a8ad0 con 0x5646a99612f0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1a7d30 con 0x5646a99694d0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc969d7a740 10 client.0 _ll_register_callbacks cb 0x7fc944009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.404+0000 7fc960c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc960c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55062/0 (socket says 172.21.15.50:55062) 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc960c00640 1 -- 172.21.15.50:0/1514376640 learned_addr learned my addr 172.21.15.50:0/1514376640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc960c00640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d44920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 -- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 msgr2=0x5646a9d434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 -- 172.21.15.50:0/1514376640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 msgr2=0x5646a9d44920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d44920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.405+0000 7fc95b400640 1 -- 172.21.15.50:0/1514376640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a9d44ee0 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc95b400640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d439f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5646a9d2d2a0 tx=0x7fc944027a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e8718fbb4e9256 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc949600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94402b070 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9440119a0 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fc94402e040 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc944011bd0 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc969d7a740 10 client.25816 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646a9d46170 con 0x5646a99694d0 2024-05-10T21:11:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.406+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9440119a0 con 0x5646a99694d0 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc944027c50 con 0x5646a99694d0 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 10 client.25816 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 20 client.25816 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a991e1c0 0x5646a9db6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 4 client.25816 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646a9c97fd0 con 0x5646a991e1c0 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.407+0000 7fc960c00640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a991e1c0 0x5646a9db6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.408+0000 7fc960c00640 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a991e1c0 0x5646a9db6980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e063bb6165ba8557 server_cookie=8ac6c4ad541bbae6 in_seq=0 out_seq=0 2024-05-10T21:11:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.408+0000 7fc949600640 10 client.25816 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.411+0000 7fc949600640 1 -- 172.21.15.50:0/1514376640 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646a9c97fd0 con 0x5646a991e1c0 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.411+0000 7fc949600640 10 client.25816 handle_command_reply: tid=0 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.411+0000 7fc969d7a740 1 client.25816 shutdown 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 msgr2=0x5646a9d439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d439f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5646a9d2d2a0 tx=0x7fc944027a30 comp rx=0 tx=0).stop 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 shutdown_connections 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646a991e1c0 0x5646a9db6980 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646a99694d0 0x5646a9d439f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646a998a1b0 0x5646a9d44920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 --2- 172.21.15.50:0/1514376640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a99612f0 0x5646a9d434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 >> 172.21.15.50:0/1514376640 conn(0x5646a997e300 msgr2=0x5646a9d270f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc949600640 0 client.25816 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 shutdown_connections 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 1 -- 172.21.15.50:0/1514376640 wait complete. 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.412+0000 7fc969d7a740 20 client.25816 trim_cache size 0 max 16384 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 msgr2=0x7fc93c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 0x7fc93c0b4d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5646a9d2d4e0 tx=0x7fc94c0e8060 comp rx=0 tx=0).stop 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 msgr2=0x7fc95c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e9f0 tx=0x7fc94c00eeb0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 shutdown_connections 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc93c0b2860 0x7fc93c0b4d70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc95c107120 0x7fc95c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc95c10be20 0x7fc95c1a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 --2- 172.21.15.50:0/3555004414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc95c105ee0 0x7fc95c19c4b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 >> 172.21.15.50:0/3555004414 conn(0x7fc95c1010b0 msgr2=0x7fc95c107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 shutdown_connections 2024-05-10T21:11:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:00.413+0000 7fc962c00640 1 -- 172.21.15.50:0/3555004414 wait complete. 2024-05-10T21:11:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.001+0000 7f30cbc00700 1 -- 172.21.15.50:0/879922175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 msgr2=0x7f30c4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.001+0000 7f30cbc00700 1 --2- 172.21.15.50:0/879922175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c4106980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009990 tx=0x7f30b0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 -- 172.21.15.50:0/879922175 shutdown_connections 2024-05-10T21:11:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 --2- 172.21.15.50:0/879922175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c410c5b0 0x7f30c410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 --2- 172.21.15.50:0/879922175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c4106ec0 0x7f30c4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 --2- 172.21.15.50:0/879922175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c4106980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 -- 172.21.15.50:0/879922175 >> 172.21.15.50:0/879922175 conn(0x7f30c4102050 msgr2=0x7f30c41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 -- 172.21.15.50:0/879922175 shutdown_connections 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.002+0000 7f30cbc00700 1 -- 172.21.15.50:0/879922175 wait complete. 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.003+0000 7f30cbc00700 1 Processor -- start 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.003+0000 7f30cbc00700 1 -- start start 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 0x7f30c419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c410c5b0 0x7f30c41a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41166e0 con 0x7f30c4106570 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4116560 con 0x7f30c4106ec0 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30cbc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4116860 con 0x7f30c410c5b0 2024-05-10T21:11:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30c9200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30c9200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60590/0 (socket says 172.21.15.50:60590) 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30c9200700 1 -- 172.21.15.50:0/3540386912 learned_addr learned my addr 172.21.15.50:0/3540386912 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30c3e00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 0x7f30c419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.004+0000 7f30c9c00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c410c5b0 0x7f30c41a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 -- 172.21.15.50:0/3540386912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c410c5b0 msgr2=0x7f30c41a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c410c5b0 0x7f30c41a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 -- 172.21.15.50:0/3540386912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 msgr2=0x7f30c419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 0x7f30c419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 -- 172.21.15.50:0/3540386912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c41a1f00 con 0x7f30c4106570 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c3e00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 0x7f30c419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c9200700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e970 tx=0x7f30b800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8004a30 con 0x7f30c4106570 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c41a21f0 con 0x7f30c4106570 2024-05-10T21:11:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b800cc90 con 0x7f30c4106570 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8005610 con 0x7f30c4106570 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.005+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c41a9b10 con 0x7f30c4106570 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.007+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40512c0 con 0x7f30c4106570 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.009+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f30b800c7e0 con 0x7f30c4106570 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:11:00.722 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.011+0000 7f30c1600700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 0x7f30ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.011+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f30b80e2590 con 0x7f30c4106570 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.011+0000 7f30c3e00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 0x7f30ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.012+0000 7f30c3e00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 0x7f30ac0b4e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30b0006010 tx=0x7f30b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.014+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30b8054df0 con 0x7f30c4106570 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.314+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c419e390 con 0x7f30c4106570 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.316+0000 7f30c1600700 1 -- 172.21.15.50:0/3540386912 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f30b80aa280 con 0x7f30c4106570 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.319+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 msgr2=0x7f30ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.319+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 0x7f30ac0b4e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30b0006010 tx=0x7f30b000b260 comp rx=0 tx=0).stop 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 msgr2=0x7f30c419d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e970 tx=0x7f30b800ec80 comp rx=0 tx=0).stop 2024-05-10T21:11:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 shutdown_connections 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f30ac0b2900 0x7f30ac0b4e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30c410c5b0 0x7f30c41a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30c4106570 0x7f30c419d030 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 --2- 172.21.15.50:0/3540386912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c4106ec0 0x7f30c419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 >> 172.21.15.50:0/3540386912 conn(0x7f30c4102050 msgr2=0x7f30c41036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 shutdown_connections 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:00.320+0000 7f30cbc00700 1 -- 172.21.15.50:0/3540386912 wait complete. 2024-05-10T21:11:00.724 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:11:01.019 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:00 smithi050 ceph-mon[29546]: pgmap v524: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:11:01.019 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:00 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3540386912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:01.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:00 smithi092 ceph-mon[34415]: pgmap v524: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:11:01.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:00 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3540386912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:00 smithi142 ceph-mon[34846]: pgmap v524: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:11:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:00 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3540386912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:01.424 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id 31fca0c0 --release --await' 2024-05-10T21:11:01.425 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db --set-id 31fca0c0 --release --await 2024-05-10T21:11:02.021 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:02.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.070+0000 7f6b20800700 1 -- 172.21.15.50:0/261796971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 msgr2=0x7f6b18110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.070+0000 7f6b20800700 1 --2- 172.21.15.50:0/261796971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b18110660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009990 tx=0x7f6b08009ca0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 -- 172.21.15.50:0/261796971 shutdown_connections 2024-05-10T21:11:02.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 --2- 172.21.15.50:0/261796971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18075d30 0x7f6b18076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 --2- 172.21.15.50:0/261796971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18110ba0 0x7f6b18113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 --2- 172.21.15.50:0/261796971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b18110660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 -- 172.21.15.50:0/261796971 >> 172.21.15.50:0/261796971 conn(0x7f6b18100040 msgr2=0x7f6b18102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 -- 172.21.15.50:0/261796971 shutdown_connections 2024-05-10T21:11:02.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.071+0000 7f6b20800700 1 -- 172.21.15.50:0/261796971 wait complete. 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.072+0000 7f6b20800700 1 Processor -- start 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 -- start start 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b1819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18110ba0 0x7f6b181a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181166f0 con 0x7f6b18076680 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18116570 con 0x7f6b18110ba0 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.073+0000 7f6b20800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18116870 con 0x7f6b18075d30 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43824/0 (socket says 172.21.15.50:43824) 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 -- 172.21.15.50:0/3838022671 learned_addr learned my addr 172.21.15.50:0/3838022671 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:02.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1e800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18110ba0 0x7f6b181a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1d400700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b1819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 -- 172.21.15.50:0/3838022671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18110ba0 msgr2=0x7f6b181a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18110ba0 0x7f6b181a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 -- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 msgr2=0x7f6b1819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b1819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1de00700 1 -- 172.21.15.50:0/3838022671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b181a1f50 con 0x7f6b18075d30 2024-05-10T21:11:02.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.074+0000 7f6b1d400700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b1819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:02.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9a0 tx=0x7f6b0c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b12a00700 1 -- 172.21.15.50:0/3838022671 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cbd0 con 0x7f6b18075d30 2024-05-10T21:11:02.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181a2240 con 0x7f6b18075d30 2024-05-10T21:11:02.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b12a00700 1 -- 172.21.15.50:0/3838022671 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c00cd70 con 0x7f6b18075d30 2024-05-10T21:11:02.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181a9b60 con 0x7f6b18075d30 2024-05-10T21:11:02.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.075+0000 7f6b12a00700 1 -- 172.21.15.50:0/3838022671 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c018680 con 0x7f6b18075d30 2024-05-10T21:11:02.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.079+0000 7f6b12a00700 1 -- 172.21.15.50:0/3838022671 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6b0c018820 con 0x7f6b18075d30 2024-05-10T21:11:02.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.081+0000 7f6b2f2e6b80 1 Processor -- start 2024-05-10T21:11:02.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.081+0000 7f6b2f2e6b80 1 -- start start 2024-05-10T21:11:02.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013974e40 0x556013af82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013af24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013af2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556013a14a20 con 0x556013af8830 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5560139b2760 con 0x556013af29f0 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556013a109e0 con 0x556013974e40 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b12a00700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 0x7f6b000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013af24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013af24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60662/0 (socket says 172.21.15.50:60662) 2024-05-10T21:11:02.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b1de00700 1 -- 172.21.15.50:0/752923962 learned_addr learned my addr 172.21.15.50:0/752923962 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:02.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.082+0000 7f6b1e800700 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013974e40 0x556013af82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013af2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1e800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 0x7f6b000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b12a00700 1 -- 172.21.15.50:0/3838022671 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6b0c0e2510 con 0x7f6b18075d30 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 -- 172.21.15.50:0/752923962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013974e40 msgr2=0x556013af82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013974e40 0x556013af82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 -- 172.21.15.50:0/752923962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 msgr2=0x556013af24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013af24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 -- 172.21.15.50:0/752923962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560139618d0 con 0x556013af29f0 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.083+0000 7f6b1d400700 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013af2e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b18103490 tx=0x7f6b080096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e1cb7c9668deda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.084+0000 7f6b1e800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 0x7f6b000b4d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009ec0 tx=0x7f6b1400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.084+0000 7f6b07400700 1 -- 172.21.15.50:0/752923962 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080054a0 con 0x556013af29f0 2024-05-10T21:11:02.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.084+0000 7f6b07400700 1 -- 172.21.15.50:0/752923962 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08005640 con 0x556013af29f0 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.084+0000 7f6b07400700 1 -- 172.21.15.50:0/752923962 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003c90 con 0x556013af29f0 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/752923962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 msgr2=0x556013af2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013af2e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b18103490 tx=0x7f6b080096c0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/752923962 shutdown_connections 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013af2e00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013974e40 0x556013af82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/752923962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013af24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/752923962 >> 172.21.15.50:0/752923962 conn(0x5560139b9f70 msgr2=0x556013aed2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/752923962 shutdown_connections 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.085+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/752923962 wait complete. 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.086+0000 7f6b2f2e6b80 1 -- start start 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.086+0000 7f6b2f2e6b80 1 Processor -- start 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013974e40 0x556013afbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013afc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556013aff2b0 con 0x556013974e40 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55601397ab20 con 0x556013af29f0 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556013aff430 con 0x556013af8830 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b2f2e6b80 10 client.0 _ll_register_callbacks cb 0x5560139bbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b1e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b1e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43846/0 (socket says 172.21.15.50:43846) 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b1e800700 1 -- 172.21.15.50:0/2303180009 learned_addr learned my addr 172.21.15.50:0/2303180009 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b1d400700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013afc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.087+0000 7f6b1de00700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013974e40 0x556013afbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 -- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 msgr2=0x556013afc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013afc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 -- 172.21.15.50:0/2303180009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013974e40 msgr2=0x556013afbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013974e40 0x556013afbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 -- 172.21.15.50:0/2303180009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560139da420 con 0x556013af8830 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1d400700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013afc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b1e800700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b1401a7a0 tx=0x7f6b14004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa96bd8272d4cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6afbe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:02.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14008240 con 0x556013af8830 2024-05-10T21:11:02.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b08005640 con 0x556013af8830 2024-05-10T21:11:02.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.088+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6b14012040 con 0x556013af8830 2024-05-10T21:11:02.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.089+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b140083e0 con 0x556013af8830 2024-05-10T21:11:02.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.089+0000 7f6b2f2e6b80 10 client.25828 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.089+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556013b09e10 con 0x556013af8830 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.089+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08005640 con 0x556013af8830 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6b14007890 con 0x556013af8830 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 20 client.25828 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b0c3a0 0x556013b0e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 4 client.25828 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.090+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556013abf1c0 con 0x556013b0c3a0 2024-05-10T21:11:02.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.091+0000 7f6b1de00700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b0c3a0 0x556013b0e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.091+0000 7f6b1de00700 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b0c3a0 0x556013b0e7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf0f50e8b10a466 server_cookie=a1420ca6e9938e3a in_seq=0 out_seq=0 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.091+0000 7f6afbe00700 10 client.25828 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.092+0000 7f6afbe00700 1 -- 172.21.15.50:0/2303180009 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556013abf1c0 con 0x556013b0c3a0 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.092+0000 7f6afbe00700 10 client.25828 handle_command_reply: tid=0 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.092+0000 7f6b2f2e6b80 1 client.25828 shutdown 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.092+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 msgr2=0x556013afc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.092+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b1401a7a0 tx=0x7f6b14004ad0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 shutdown_connections 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b0c3a0 0x556013b0e7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013afc9e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013974e40 0x556013afbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/2303180009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013af29f0 0x556013afc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6afbe00700 0 client.25828 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 >> 172.21.15.50:0/2303180009 conn(0x5560139b9f70 msgr2=0x556013af14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 shutdown_connections 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/2303180009 wait complete. 2024-05-10T21:11:02.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.093+0000 7f6b2f2e6b80 20 client.25828 trim_cache size 0 max 16384 2024-05-10T21:11:02.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 Processor -- start 2024-05-10T21:11:02.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 -- start start 2024-05-10T21:11:02.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013af24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013ae70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b3770 con 0x556013af8830 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b1000 con 0x556013974e40 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.110+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181aee90 con 0x556013af29f0 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1e800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013ae70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013af24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:43860/0 (socket says 172.21.15.50:43860) 2024-05-10T21:11:02.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 -- 172.21.15.50:0/515337232 learned_addr learned my addr 172.21.15.50:0/515337232 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:02.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013af24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55098/0 (socket says 172.21.15.50:55098) 2024-05-10T21:11:02.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 -- 172.21.15.50:0/515337232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 msgr2=0x556013af24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013af24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 -- 172.21.15.50:0/515337232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 msgr2=0x556013ae70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013ae70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.111+0000 7f6b1d400700 1 -- 172.21.15.50:0/515337232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556013aedda0 con 0x556013af29f0 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.112+0000 7f6b1d400700 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009750 tx=0x7f6b080054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec696f304a869a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.112+0000 7f6b06a00700 1 -- 172.21.15.50:0/515337232 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031070 con 0x556013af29f0 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.112+0000 7f6b06a00700 1 -- 172.21.15.50:0/515337232 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08005690 con 0x556013af29f0 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.112+0000 7f6b06a00700 1 -- 172.21.15.50:0/515337232 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08027040 con 0x556013af29f0 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/515337232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 msgr2=0x556013af63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009750 tx=0x7f6b080054a0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/515337232 shutdown_connections 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013af24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af8830 0x556013ae70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/515337232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af29f0 0x556013af63c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/515337232 >> 172.21.15.50:0/515337232 conn(0x5560139b9f70 msgr2=0x556013af5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/515337232 shutdown_connections 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.113+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/515337232 wait complete. 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.114+0000 7f6b2f2e6b80 1 -- start start 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.114+0000 7f6b2f2e6b80 1 Processor -- start 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.114+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013b13c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013b111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181af4e0 con 0x556013af29f0 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b0930 con 0x556013974e40 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181afb90 con 0x556013af8830 2024-05-10T21:11:02.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b2f2e6b80 10 client.0 _ll_register_callbacks cb 0x5560139bbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60692/0 (socket says 172.21.15.50:60692) 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 -- 172.21.15.50:0/3058035373 learned_addr learned my addr 172.21.15.50:0/3058035373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013b13c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1e800700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013b111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 -- 172.21.15.50:0/3058035373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 msgr2=0x556013b111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013b111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 -- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 msgr2=0x556013b13c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013b13c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1d400700 1 -- 172.21.15.50:0/3058035373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556013b11850 con 0x556013af29f0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.115+0000 7f6b1e800700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013b111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.116+0000 7f6b1d400700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003940 tx=0x7f6b08003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ecb4b182e70d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.116+0000 7f6afbe00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:11:02.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.116+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08031040 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.116+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f6b08003680 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.116+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b08005690 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b0800b720 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 10 client.15735 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556013b76360 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08005690 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6b0802a030 con 0x556013af29f0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 20 client.15735 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b76530 0x556013b78950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 4 client.15735 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.117+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556013a22400 con 0x556013b76530 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b76530 0x556013b78950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:02.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6b1de00700 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b76530 0x556013b78950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca63755ce5d1e3f9 server_cookie=8939ba3334fa440b in_seq=0 out_seq=0 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6afbe00700 10 client.15735 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6afbe00700 1 -- 172.21.15.50:0/3058035373 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556013a22400 con 0x556013b76530 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6afbe00700 10 client.15735 handle_command_reply: tid=0 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.118+0000 7f6b2f2e6b80 1 client.15735 shutdown 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 msgr2=0x556013b14140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003940 tx=0x7f6b08003e90 comp rx=0 tx=0).stop 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 shutdown_connections 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556013b76530 0x556013b78950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556013974e40 0x556013b13c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556013af8830 0x556013b111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 --2- 172.21.15.50:0/3058035373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556013af29f0 0x556013b14140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6afbe00700 0 client.15735 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:02.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 >> 172.21.15.50:0/3058035373 conn(0x5560139b9f70 msgr2=0x556013af7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 shutdown_connections 2024-05-10T21:11:02.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 1 -- 172.21.15.50:0/3058035373 wait complete. 2024-05-10T21:11:02.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.119+0000 7f6b2f2e6b80 20 client.15735 trim_cache size 0 max 16384 2024-05-10T21:11:02.123 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:11:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 msgr2=0x7f6b000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 0x7f6b000b4d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009ec0 tx=0x7f6b1400b040 comp rx=0 tx=0).stop 2024-05-10T21:11:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 msgr2=0x7f6b1819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9a0 tx=0x7f6b0c00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 shutdown_connections 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6b000b2880 0x7f6b000b4d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b18110ba0 0x7f6b181a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b18075d30 0x7f6b1819d080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 --2- 172.21.15.50:0/3838022671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b18076680 0x7f6b1819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 >> 172.21.15.50:0/3838022671 conn(0x7f6b18100040 msgr2=0x7f6b181006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 shutdown_connections 2024-05-10T21:11:02.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:02.120+0000 7f6b20800700 1 -- 172.21.15.50:0/3838022671 wait complete. 2024-05-10T21:11:03.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:02 smithi092 ceph-mon[34415]: pgmap v525: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:11:03.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:02 smithi142 ceph-mon[34846]: pgmap v525: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:11:03.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:02 smithi050 ceph-mon[29546]: pgmap v525: 97 pgs: 97 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:11:04.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:11:04.746 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:11:05.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:04 smithi092 ceph-mon[34415]: pgmap v526: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:11:05.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:04 smithi142 ceph-mon[34846]: pgmap v526: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:11:05.132 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:04 smithi050 ceph-mon[29546]: pgmap v526: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.132+0000 7faa9d200700 1 -- 172.21.15.50:0/1286624778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa98106590 msgr2=0x7faa981069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.132+0000 7faa9d200700 1 --2- 172.21.15.50:0/1286624778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa98106590 0x7faa981069a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faa84009990 tx=0x7faa84009ca0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 -- 172.21.15.50:0/1286624778 shutdown_connections 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 --2- 172.21.15.50:0/1286624778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106ee0 0x7faa98113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 --2- 172.21.15.50:0/1286624778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa9810c5d0 0x7faa9810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 --2- 172.21.15.50:0/1286624778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa98106590 0x7faa981069a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 -- 172.21.15.50:0/1286624778 >> 172.21.15.50:0/1286624778 conn(0x7faa98102090 msgr2=0x7faa981044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 -- 172.21.15.50:0/1286624778 shutdown_connections 2024-05-10T21:11:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.133+0000 7faa9d200700 1 -- 172.21.15.50:0/1286624778 wait complete. 2024-05-10T21:11:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.134+0000 7faa9d200700 1 Processor -- start 2024-05-10T21:11:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 -- start start 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106590 0x7faa98110780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa9810c5d0 0x7faa98111200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa98116720 con 0x7faa9810c5d0 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa981165a0 con 0x7faa98106590 2024-05-10T21:11:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.135+0000 7faa9d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa981168a0 con 0x7faa98106ee0 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa96800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa9810c5d0 0x7faa98111200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44926/0 (socket says 172.21.15.50:44926) 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 -- 172.21.15.50:0/743734982 learned_addr learned my addr 172.21.15.50:0/743734982 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95e00700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106590 0x7faa98110780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 -- 172.21.15.50:0/743734982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106590 msgr2=0x7faa98110780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106590 0x7faa98110780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 -- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa9810c5d0 msgr2=0x7faa98111200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa9810c5d0 0x7faa98111200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 -- 172.21.15.50:0/743734982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981ad5f0 con 0x7faa98106ee0 2024-05-10T21:11:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.136+0000 7faa95400700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa840098e0 tx=0x7faa840047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.137+0000 7faa8ea00700 1 -- 172.21.15.50:0/743734982 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84026070 con 0x7faa98106ee0 2024-05-10T21:11:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.137+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981ad880 con 0x7faa98106ee0 2024-05-10T21:11:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.137+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981add30 con 0x7faa98106ee0 2024-05-10T21:11:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.137+0000 7faa8ea00700 1 -- 172.21.15.50:0/743734982 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa84009510 con 0x7faa98106ee0 2024-05-10T21:11:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.137+0000 7faa8ea00700 1 -- 172.21.15.50:0/743734982 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84021da0 con 0x7faa98106ee0 2024-05-10T21:11:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.142+0000 7faa8ea00700 1 -- 172.21.15.50:0/743734982 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faa84009d10 con 0x7faa98106ee0 2024-05-10T21:11:05.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.143+0000 7faa8ea00700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 0x7faa7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.143+0000 7faa8ea00700 1 -- 172.21.15.50:0/743734982 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7faa840e2c10 con 0x7faa98106ee0 2024-05-10T21:11:05.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.144+0000 7faa95e00700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 0x7faa7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.144+0000 7faaabc7ab80 1 Processor -- start 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.144+0000 7faa95e00700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 0x7faa7c0b4d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faa98107a00 tx=0x7faa88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 -- start start 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536600 0x5585af540cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af543210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5585af458b20 con 0x5585af536600 2024-05-10T21:11:05.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585af3f6860 con 0x5585af536a10 2024-05-10T21:11:05.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.145+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585af454ae0 con 0x5585af3b8f40 2024-05-10T21:11:05.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44928/0 (socket says 172.21.15.50:44928) 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 -- 172.21.15.50:0/800775591 learned_addr learned my addr 172.21.15.50:0/800775591 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95400700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536600 0x5585af540cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa96800700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af543210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 -- 172.21.15.50:0/800775591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 msgr2=0x5585af543210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af543210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 -- 172.21.15.50:0/800775591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536600 msgr2=0x5585af540cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536600 0x5585af540cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa95e00700 1 -- 172.21.15.50:0/800775591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585af3a59d0 con 0x5585af3b8f40 2024-05-10T21:11:05.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.146+0000 7faa96800700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af543210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:05.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.147+0000 7faa95e00700 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faa88011da0 tx=0x7faa88014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69b2da3913893d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.147+0000 7faa83400700 1 -- 172.21.15.50:0/800775591 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88004a30 con 0x5585af3b8f40 2024-05-10T21:11:05.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.147+0000 7faa83400700 1 -- 172.21.15.50:0/800775591 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa880054d0 con 0x5585af3b8f40 2024-05-10T21:11:05.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.147+0000 7faa83400700 1 -- 172.21.15.50:0/800775591 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88003c60 con 0x5585af3b8f40 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.148+0000 7faaabc7ab80 1 -- 172.21.15.50:0/800775591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 msgr2=0x5585af5360c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.148+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faa88011da0 tx=0x7faa88014000 comp rx=0 tx=0).stop 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 -- 172.21.15.50:0/800775591 shutdown_connections 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af543210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af5360c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/800775591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536600 0x5585af540cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 -- 172.21.15.50:0/800775591 >> 172.21.15.50:0/800775591 conn(0x5585af3f7b80 msgr2=0x5585af531f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 -- 172.21.15.50:0/800775591 shutdown_connections 2024-05-10T21:11:05.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.149+0000 7faaabc7ab80 1 -- 172.21.15.50:0/800775591 wait complete. 2024-05-10T21:11:05.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.150+0000 7faaabc7ab80 1 -- start start 2024-05-10T21:11:05.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.150+0000 7faaabc7ab80 1 Processor -- start 2024-05-10T21:11:05.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af540d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af5412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5585af543b80 con 0x5585af536a10 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585af3bec20 con 0x5585af3b8f40 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585af543d00 con 0x5585af536600 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faaabc7ab80 10 client.0 _ll_register_callbacks cb 0x5585af3ec480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faa95e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faa95e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56348/0 (socket says 172.21.15.50:56348) 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faa95e00700 1 -- 172.21.15.50:0/4149502135 learned_addr learned my addr 172.21.15.50:0/4149502135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.151+0000 7faa95400700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af540d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa96800700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af5412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 -- 172.21.15.50:0/4149502135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 msgr2=0x5585af540d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af540d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 -- 172.21.15.50:0/4149502135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 msgr2=0x5585af5412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af5412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 -- 172.21.15.50:0/4149502135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585af41e520 con 0x5585af3b8f40 2024-05-10T21:11:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95400700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af540d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa95e00700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa88004910 tx=0x7faa88004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80eb1a3b9264577b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.152+0000 7faa77e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88005680 con 0x5585af3b8f40 2024-05-10T21:11:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa880054d0 con 0x5585af3b8f40 2024-05-10T21:11:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7faa88003860 con 0x5585af3b8f40 2024-05-10T21:11:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa88003a60 con 0x5585af3b8f40 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faaabc7ab80 10 client.25874 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585af54e8d0 con 0x5585af3b8f40 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.153+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa880054d0 con 0x5585af3b8f40 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faa88031c70 con 0x5585af3b8f40 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 10 client.25874 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 10 client.25874 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 20 client.25874 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af550e00 0x5585af553220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 4 client.25874 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585af503170 con 0x5585af550e00 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.154+0000 7faa95400700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af550e00 0x5585af553220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.155+0000 7faa95400700 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af550e00 0x5585af553220 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bb10eb43bedfa2 server_cookie=cfb12bac3293093f in_seq=0 out_seq=0 2024-05-10T21:11:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.155+0000 7faa77e00700 10 client.25874 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faa77e00700 1 -- 172.21.15.50:0/4149502135 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5585af503170 con 0x5585af550e00 2024-05-10T21:11:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faa77e00700 10 client.25874 handle_command_reply: tid=0 2024-05-10T21:11:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 client.25874 shutdown 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 msgr2=0x5585af540830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa88004910 tx=0x7faa88004940 comp rx=0 tx=0).stop 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 shutdown_connections 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af550e00 0x5585af553220 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af3b8f40 0x5585af540830 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af5412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/4149502135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af540d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faa77e00700 0 client.25874 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 >> 172.21.15.50:0/4149502135 conn(0x5585af3f7b80 msgr2=0x5585af5311e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 shutdown_connections 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.156+0000 7faaabc7ab80 1 -- 172.21.15.50:0/4149502135 wait complete. 2024-05-10T21:11:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.157+0000 7faaabc7ab80 20 client.25874 trim_cache size 0 max 16384 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.179+0000 7faaabc7ab80 1 Processor -- start 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.179+0000 7faaabc7ab80 1 -- start start 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af53f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af53a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800fae0 con 0x5585af536a10 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800d370 con 0x5585af536600 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800b1c0 con 0x5585af3b8f40 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa96800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af53a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa95400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af53f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa96800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af53a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34132/0 (socket says 172.21.15.50:34132) 2024-05-10T21:11:05.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa95400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56362/0 (socket says 172.21.15.50:56362) 2024-05-10T21:11:05.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.180+0000 7faa96800700 1 -- 172.21.15.50:0/3003980229 learned_addr learned my addr 172.21.15.50:0/3003980229 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:05.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 -- 172.21.15.50:0/3003980229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 msgr2=0x5585af53f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af53f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 -- 172.21.15.50:0/3003980229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 msgr2=0x5585af53a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af53a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 -- 172.21.15.50:0/3003980229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585af533540 con 0x5585af536600 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95e00700 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af53f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa95400700 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa8400f740 tx=0x7faa84023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554126de7bba75dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa82a00700 1 -- 172.21.15.50:0/3003980229 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840ae640 con 0x5585af536600 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa82a00700 1 -- 172.21.15.50:0/3003980229 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa840b3060 con 0x5585af536600 2024-05-10T21:11:05.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.181+0000 7faa82a00700 1 -- 172.21.15.50:0/3003980229 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84021670 con 0x5585af536600 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 -- 172.21.15.50:0/3003980229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 msgr2=0x5585af542c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa8400f740 tx=0x7faa84023040 comp rx=0 tx=0).stop 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 -- 172.21.15.50:0/3003980229 shutdown_connections 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af536a10 0x5585af53a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af3b8f40 0x5585af53f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/3003980229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536600 0x5585af542c30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.182+0000 7faaabc7ab80 1 -- 172.21.15.50:0/3003980229 >> 172.21.15.50:0/3003980229 conn(0x5585af3f7b80 msgr2=0x5585af53b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.183+0000 7faaabc7ab80 1 -- 172.21.15.50:0/3003980229 shutdown_connections 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.183+0000 7faaabc7ab80 1 -- 172.21.15.50:0/3003980229 wait complete. 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.183+0000 7faaabc7ab80 1 -- start start 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.183+0000 7faaabc7ab80 1 Processor -- start 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af553a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af553fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800b830 con 0x5585af3b8f40 2024-05-10T21:11:05.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800cca0 con 0x5585af536a10 2024-05-10T21:11:05.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800bf00 con 0x5585af536600 2024-05-10T21:11:05.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faaabc7ab80 10 client.0 _ll_register_callbacks cb 0x5585af3ec480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faa95e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faa95e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34148/0 (socket says 172.21.15.50:34148) 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.184+0000 7faa95e00700 1 -- 172.21.15.50:0/1050970247 learned_addr learned my addr 172.21.15.50:0/1050970247 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af553fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa95400700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af553a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 -- 172.21.15.50:0/1050970247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 msgr2=0x5585af553a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af553a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 -- 172.21.15.50:0/1050970247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 msgr2=0x5585af556700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:05.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa96800700 1 -- 172.21.15.50:0/1050970247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585af554670 con 0x5585af536a10 2024-05-10T21:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa95e00700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.185+0000 7faa95400700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af553a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faa96800700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af553fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa90009790 tx=0x7faa90009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a5c226124751c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:05.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faa77e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90004bb0 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7faa90004d50 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa840b3060 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.186+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa90017820 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 10 client.25880 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585af5bb230 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa840b3060 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faa900046c0 con 0x5585af536a10 2024-05-10T21:11:05.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 10 client.25880 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 10 client.25880 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 20 client.25880 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af5bb940 0x5585af5bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 4 client.25880 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "31fca0c0", "release": true, "await": true}] 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.187+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "set_id": "31fca0c0", "release": true, "await": true}) -- 0x5585af466500 con 0x5585af5bb940 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.188+0000 7faa95e00700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af5bb940 0x5585af5bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.188+0000 7faa95e00700 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af5bb940 0x5585af5bdd60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b95306bf11fdaa server_cookie=8efedd8d410d8ffa in_seq=0 out_seq=0 2024-05-10T21:11:05.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.188+0000 7faa77e00700 10 client.25880 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.191+0000 7faa77e00700 1 -- 172.21.15.50:0/1050970247 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5585af466500 con 0x5585af5bb940 2024-05-10T21:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.191+0000 7faa77e00700 10 client.25880 handle_command_reply: tid=0 2024-05-10T21:11:05.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.191+0000 7faaabc7ab80 1 client.25880 shutdown 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 msgr2=0x5585af553fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af553fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa90009790 tx=0x7faa90009300 comp rx=0 tx=0).stop 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 shutdown_connections 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5585af5bb940 0x5585af5bdd60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585af536a10 0x5585af553fc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585af3b8f40 0x5585af556700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 --2- 172.21.15.50:0/1050970247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585af536600 0x5585af553a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 >> 172.21.15.50:0/1050970247 conn(0x5585af3f7b80 msgr2=0x5585af5400e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faa77e00700 0 client.25880 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 shutdown_connections 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.192+0000 7faaabc7ab80 1 -- 172.21.15.50:0/1050970247 wait complete. 2024-05-10T21:11:05.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.193+0000 7faaabc7ab80 20 client.25880 trim_cache size 0 max 16384 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 25, 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 8, 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "31fca0c0": { 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 8, 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:11:05.198 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 81.3, 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:05.199 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:11:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.193+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 msgr2=0x7faa7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.193+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 0x7faa7c0b4d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faa98107a00 tx=0x7faa88009260 comp rx=0 tx=0).stop 2024-05-10T21:11:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.193+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 msgr2=0x7faa98110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.193+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa840098e0 tx=0x7faa840047b0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 shutdown_connections 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7faa7c0b2880 0x7faa7c0b4d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa98106590 0x7faa98110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa9810c5d0 0x7faa98111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 --2- 172.21.15.50:0/743734982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa98106ee0 0x7faa98110cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 >> 172.21.15.50:0/743734982 conn(0x7faa98102090 msgr2=0x7faa981047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 shutdown_connections 2024-05-10T21:11:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:05.194+0000 7faa9d200700 1 -- 172.21.15.50:0/743734982 wait complete. 2024-05-10T21:11:05.207 INFO:tasks.workunit.client.0.smithi050.stderr:+ echo 'getting blogbench' 2024-05-10T21:11:05.208 INFO:tasks.workunit.client.0.smithi050.stdout:getting blogbench 2024-05-10T21:11:05.208 INFO:tasks.workunit.client.0.smithi050.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-05-10T21:11:05.224 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:05.240 INFO:tasks.workunit.client.0.smithi050.stderr:--2024-05-10 21:11:05-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-05-10T21:11:05.242 INFO:tasks.workunit.client.0.smithi050.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-10T21:11:05.281 INFO:tasks.workunit.client.0.smithi050.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-10T21:11:05.319 INFO:tasks.workunit.client.0.smithi050.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-10T21:11:05.319 INFO:tasks.workunit.client.0.smithi050.stderr:Length: 103355 (101K) [application/octet-stream] 2024-05-10T21:11:05.321 INFO:tasks.workunit.client.0.smithi050.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-05-10T21:11:05.327 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:11:05.397 INFO:tasks.workunit.client.0.smithi050.stderr: 0K .......... .......... .......... .......... .......... 49% 669K 0s 2024-05-10T21:11:05.436 INFO:tasks.workunit.client.0.smithi050.stderr: 50K .......... .......... .......... .......... .......... 99% 1.24M 0s 2024-05-10T21:11:05.437 INFO:tasks.workunit.client.0.smithi050.stderr: 100K 100% 1.74T=0.1s 2024-05-10T21:11:05.437 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:11:05.437 INFO:tasks.workunit.client.0.smithi050.stderr:2024-05-10 21:11:05 (886 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-05-10T21:11:05.437 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-05-10T21:11:05.439 INFO:tasks.workunit.client.0.smithi050.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-05-10T21:11:05.604 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/ 2024-05-10T21:11:05.607 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/m4/ 2024-05-10T21:11:05.608 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-05-10T21:11:05.611 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/man/ 2024-05-10T21:11:05.614 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/man/blogbench.8 2024-05-10T21:11:05.615 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/man/Makefile.am 2024-05-10T21:11:05.618 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/man/Makefile.in 2024-05-10T21:11:05.619 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/ 2024-05-10T21:11:05.623 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/mysnprintf.c 2024-05-10T21:11:05.625 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/mysnprintf.h 2024-05-10T21:11:05.626 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/blogbench.c 2024-05-10T21:11:05.628 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/blogbench.h 2024-05-10T21:11:05.630 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/fakesnprintf.c 2024-05-10T21:11:05.632 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/fakesnprintf.h 2024-05-10T21:11:05.634 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/blogbench_p.h 2024-05-10T21:11:05.636 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-05-10T21:11:05.638 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-05-10T21:11:05.639 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/Makefile.am 2024-05-10T21:11:05.641 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/Makefile.in 2024-05-10T21:11:05.643 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/process.c 2024-05-10T21:11:05.645 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/globals.h 2024-05-10T21:11:05.647 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/commenter.c 2024-05-10T21:11:05.649 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/rewriter.c 2024-05-10T21:11:05.651 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/reader.c 2024-05-10T21:11:05.652 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/writer.c 2024-05-10T21:11:05.654 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/src/helpers.c 2024-05-10T21:11:05.656 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/NEWS 2024-05-10T21:11:05.658 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/compile 2024-05-10T21:11:05.660 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/depcomp 2024-05-10T21:11:05.661 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/aclocal.m4 2024-05-10T21:11:05.663 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/README 2024-05-10T21:11:05.665 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/configure 2024-05-10T21:11:05.675 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/configure.ac 2024-05-10T21:11:05.677 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/config.guess 2024-05-10T21:11:05.678 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/install-sh 2024-05-10T21:11:05.680 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/config.sub 2024-05-10T21:11:05.683 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/missing 2024-05-10T21:11:05.684 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/mkinstalldirs 2024-05-10T21:11:05.686 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/Makefile.am 2024-05-10T21:11:05.688 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/Makefile.in 2024-05-10T21:11:05.690 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/config.h.in 2024-05-10T21:11:05.691 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/AUTHORS 2024-05-10T21:11:05.693 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/INSTALL 2024-05-10T21:11:05.695 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/ChangeLog 2024-05-10T21:11:05.697 INFO:tasks.workunit.client.0.smithi050.stdout:blogbench-1.0/COPYING 2024-05-10T21:11:05.701 INFO:tasks.workunit.client.0.smithi050.stderr:+ cd blogbench-1.0/ 2024-05-10T21:11:05.701 INFO:tasks.workunit.client.0.smithi050.stdout:making blogbench 2024-05-10T21:11:05.701 INFO:tasks.workunit.client.0.smithi050.stderr:+ echo 'making blogbench' 2024-05-10T21:11:05.701 INFO:tasks.workunit.client.0.smithi050.stderr:+ ./configure 2024-05-10T21:11:06.058 INFO:tasks.workunit.client.0.smithi050.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-10T21:11:06.235 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '31fca0c0', seconds elapsed: 41.5 2024-05-10T21:11:07.080 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether build environment is sane... yes 2024-05-10T21:11:07.107 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gawk... gawk 2024-05-10T21:11:07.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:06 smithi092 ceph-mon[34415]: pgmap v527: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:11:07.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:06 smithi142 ceph-mon[34846]: pgmap v527: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:11:07.141 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether make sets $(MAKE)... yes 2024-05-10T21:11:07.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:06 smithi050 ceph-mon[29546]: pgmap v527: 97 pgs: 97 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T21:11:07.185 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc... gcc 2024-05-10T21:11:07.314 INFO:tasks.workunit.client.0.smithi050.stdout:checking for C compiler default output file name... a.out 2024-05-10T21:11:07.321 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether the C compiler works... yes 2024-05-10T21:11:07.325 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether we are cross compiling... no 2024-05-10T21:11:07.405 INFO:tasks.workunit.client.0.smithi050.stdout:checking for suffix of executables... 2024-05-10T21:11:07.493 INFO:tasks.workunit.client.0.smithi050.stdout:checking for suffix of object files... o 2024-05-10T21:11:07.593 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether we are using the GNU C compiler... yes 2024-05-10T21:11:07.707 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether gcc accepts -g... yes 2024-05-10T21:11:07.857 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc option to accept ANSI C... none needed 2024-05-10T21:11:07.964 INFO:tasks.workunit.client.0.smithi050.stdout:checking for style of include used by make... GNU 2024-05-10T21:11:08.167 INFO:tasks.workunit.client.0.smithi050.stdout:checking dependency style of gcc... gcc3 2024-05-10T21:11:08.167 INFO:tasks.workunit.client.0.smithi050.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-10T21:11:08.300 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to run the C preprocessor... gcc -E 2024-05-10T21:11:08.419 INFO:tasks.workunit.client.0.smithi050.stdout:checking for egrep... grep -E 2024-05-10T21:11:08.446 INFO:tasks.workunit.client.0.smithi050.stdout:checking for AIX... no 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 -- 172.21.15.50:0/4186437661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105dd0 msgr2=0x7f5d4c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 --2- 172.21.15.50:0/4186437661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c10e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009990 tx=0x7f5d38009ca0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 -- 172.21.15.50:0/4186437661 shutdown_connections 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 --2- 172.21.15.50:0/4186437661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 0x7f5d4c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 --2- 172.21.15.50:0/4186437661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105480 0x7f5d4c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 --2- 172.21.15.50:0/4186437661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c10e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 -- 172.21.15.50:0/4186437661 >> 172.21.15.50:0/4186437661 conn(0x7f5d4c0fef30 msgr2=0x7f5d4c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 -- 172.21.15.50:0/4186437661 shutdown_connections 2024-05-10T21:11:08.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.582+0000 7f5d53400700 1 -- 172.21.15.50:0/4186437661 wait complete. 2024-05-10T21:11:08.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.583+0000 7f5d53400700 1 Processor -- start 2024-05-10T21:11:08.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 -- start start 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105480 0x7f5d4c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 0x7f5d4c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1144a0 con 0x7f5d4c10e8f0 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c114320 con 0x7f5d4c105480 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d53400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c114620 con 0x7f5d4c105dd0 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.584+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44988/0 (socket says 172.21.15.50:44988) 2024-05-10T21:11:08.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 -- 172.21.15.50:0/907334012 learned_addr learned my addr 172.21.15.50:0/907334012 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:08.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d50a00700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105480 0x7f5d4c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d51400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 0x7f5d4c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 -- 172.21.15.50:0/907334012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105480 msgr2=0x7f5d4c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105480 0x7f5d4c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 -- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 msgr2=0x7f5d4c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 0x7f5d4c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 -- 172.21.15.50:0/907334012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c19fce0 con 0x7f5d4c105dd0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.585+0000 7f5d4be00700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d38005910 tx=0x7f5d3800b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.586+0000 7f5d49600700 1 -- 172.21.15.50:0/907334012 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38027070 con 0x7f5d4c105dd0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.586+0000 7f5d49600700 1 -- 172.21.15.50:0/907334012 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d38022470 con 0x7f5d4c105dd0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.586+0000 7f5d49600700 1 -- 172.21.15.50:0/907334012 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38017650 con 0x7f5d4c105dd0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.586+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c19ff70 con 0x7f5d4c105dd0 2024-05-10T21:11:08.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.586+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a0400 con 0x7f5d4c105dd0 2024-05-10T21:11:08.586 INFO:tasks.workunit.client.0.smithi050.stdout:checking for library containing strerror... none required 2024-05-10T21:11:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.590+0000 7f5d49600700 1 -- 172.21.15.50:0/907334012 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d3802a070 con 0x7f5d4c105dd0 2024-05-10T21:11:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.591+0000 7f5d61fc0b80 1 Processor -- start 2024-05-10T21:11:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.591+0000 7f5d61fc0b80 1 -- start start 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.591+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.591+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 0x55ac641e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 0x55ac641e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac64104aa0 con 0x55ac641e88b0 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ac640a27e0 con 0x55ac641e2a70 2024-05-10T21:11:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac64100a60 con 0x55ac64064ec0 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d49600700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 0x7f5d340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d50a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 0x55ac641e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44990/0 (socket says 172.21.15.50:44990) 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d51400700 1 -- 172.21.15.50:0/2686927291 learned_addr learned my addr 172.21.15.50:0/2686927291 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d50a00700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 0x55ac641e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34172/0 (socket says 172.21.15.50:34172) 2024-05-10T21:11:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d49600700 1 -- 172.21.15.50:0/907334012 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d380e2fc0 con 0x7f5d4c105dd0 2024-05-10T21:11:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.592+0000 7f5d4be00700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 0x55ac641e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 0x7f5d340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 -- 172.21.15.50:0/2686927291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 msgr2=0x55ac641e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 0x55ac641e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 -- 172.21.15.50:0/2686927291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 msgr2=0x55ac641e2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 0x55ac641e2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 -- 172.21.15.50:0/2686927291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac64051950 con 0x55ac64064ec0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d4be00700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 0x55ac641e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.593+0000 7f5d51400700 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007a40 tx=0x7f5d44007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de2c7862fa7e0b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.594+0000 7f5d51400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 0x7f5d340b4d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d4401c000 tx=0x7f5d4401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.594+0000 7f5d3e000700 1 -- 172.21.15.50:0/2686927291 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44017e50 con 0x55ac64064ec0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.594+0000 7f5d3e000700 1 -- 172.21.15.50:0/2686927291 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4401a920 con 0x55ac64064ec0 2024-05-10T21:11:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.594+0000 7f5d3e000700 1 -- 172.21.15.50:0/2686927291 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44023790 con 0x55ac64064ec0 2024-05-10T21:11:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/2686927291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 msgr2=0x55ac641e8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007a40 tx=0x7f5d44007d50 comp rx=0 tx=0).stop 2024-05-10T21:11:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/2686927291 shutdown_connections 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e2a70 0x55ac641e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e8370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/2686927291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e88b0 0x55ac641e2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/2686927291 >> 172.21.15.50:0/2686927291 conn(0x55ac640a9ff0 msgr2=0x55ac641dd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/2686927291 shutdown_connections 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.595+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/2686927291 wait complete. 2024-05-10T21:11:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.596+0000 7f5d61fc0b80 1 -- start start 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.596+0000 7f5d61fc0b80 1 Processor -- start 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac641ef330 con 0x55ac641e2a70 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ac6406aba0 con 0x55ac641e88b0 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac641ef4b0 con 0x55ac64064ec0 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d61fc0b80 10 client.0 _ll_register_callbacks cb 0x55ac640abcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34186/0 (socket says 172.21.15.50:34186) 2024-05-10T21:11:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d50a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.597+0000 7f5d4be00700 1 -- 172.21.15.50:0/1926402087 learned_addr learned my addr 172.21.15.50:0/1926402087 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d51400700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 -- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 msgr2=0x55ac641ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 -- 172.21.15.50:0/1926402087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 msgr2=0x55ac641ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 -- 172.21.15.50:0/1926402087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac640ca4a0 con 0x55ac64064ec0 2024-05-10T21:11:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d51400700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d4be00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d50a00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641ec0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ac641dedc0 tx=0x7f5d40011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad48cdc5585a4a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.598+0000 7f5d32a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4001e020 con 0x55ac64064ec0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d40004830 con 0x55ac64064ec0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4401a920 con 0x55ac64064ec0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d40004a30 con 0x55ac64064ec0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d61fc0b80 10 client.25858 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac641f9f50 con 0x55ac64064ec0 2024-05-10T21:11:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.599+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4401a920 con 0x55ac64064ec0 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d40019070 con 0x55ac64064ec0 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 20 client.25858 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac641fc590 0x55ac641fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 4 client.25858 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac641af240 con 0x55ac641fc590 2024-05-10T21:11:08.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.600+0000 7f5d4be00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac641fc590 0x55ac641fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.601+0000 7f5d4be00700 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac641fc590 0x55ac641fe9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e432dbcdffbf5a server_cookie=f13ebd1bd6ebc7b0 in_seq=0 out_seq=0 2024-05-10T21:11:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.601+0000 7f5d32a00700 10 client.25858 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d32a00700 1 -- 172.21.15.50:0/1926402087 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac641af240 con 0x55ac641fc590 2024-05-10T21:11:08.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d32a00700 10 client.25858 handle_command_reply: tid=0 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 client.25858 shutdown 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 msgr2=0x55ac641ec0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641ec0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ac641dedc0 tx=0x7f5d40011a10 comp rx=0 tx=0).stop 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 shutdown_connections 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac641fc590 0x55ac641fe9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641ec0f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/1926402087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d32a00700 0 client.25858 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.602+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 >> 172.21.15.50:0/1926402087 conn(0x55ac640a9ff0 msgr2=0x55ac641e1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.603+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 shutdown_connections 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.603+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/1926402087 wait complete. 2024-05-10T21:11:08.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.603+0000 7f5d61fc0b80 20 client.25858 trim_cache size 0 max 16384 2024-05-10T21:11:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.626+0000 7f5d61fc0b80 1 Processor -- start 2024-05-10T21:11:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.626+0000 7f5d61fc0b80 1 -- start start 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1b3550 con 0x55ac641e2a70 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1b0de0 con 0x55ac641e88b0 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1aec90 con 0x55ac64064ec0 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d4be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34198/0 (socket says 172.21.15.50:34198) 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d4be00700 1 -- 172.21.15.50:0/99524961 learned_addr learned my addr 172.21.15.50:0/99524961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d50a00700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.627+0000 7f5d51400700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 -- 172.21.15.50:0/99524961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 msgr2=0x55ac641e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 -- 172.21.15.50:0/99524961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 msgr2=0x55ac641ec9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ec9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 -- 172.21.15.50:0/99524961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac641d5460 con 0x55ac641e2a70 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d50a00700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d4be00700 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d380a78b0 tx=0x7f5d380e0960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4599fa63e9a4dae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d3d600700 1 -- 172.21.15.50:0/99524961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380ae9f0 con 0x55ac641e2a70 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d3d600700 1 -- 172.21.15.50:0/99524961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d380b3410 con 0x55ac641e2a70 2024-05-10T21:11:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.628+0000 7f5d3d600700 1 -- 172.21.15.50:0/99524961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380b2f80 con 0x55ac641e2a70 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/99524961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 msgr2=0x55ac641d79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d380a78b0 tx=0x7f5d380e0960 comp rx=0 tx=0).stop 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/99524961 shutdown_connections 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac64064ec0 0x55ac641e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.629+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/99524961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac641e2a70 0x55ac641d79e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.630+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/99524961 >> 172.21.15.50:0/99524961 conn(0x55ac640a9ff0 msgr2=0x55ac641ed580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.630+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/99524961 shutdown_connections 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.630+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/99524961 wait complete. 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.630+0000 7f5d61fc0b80 1 -- start start 2024-05-10T21:11:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 Processor -- start 2024-05-10T21:11:08.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac64064ec0 0x55ac641f4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 0x55ac641f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1af2e0 con 0x55ac64064ec0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1b0710 con 0x55ac641e88b0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c1af970 con 0x55ac641e2a70 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d61fc0b80 10 client.0 _ll_register_callbacks cb 0x55ac640abcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.631+0000 7f5d51400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d50a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac64064ec0 0x55ac641f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56444/0 (socket says 172.21.15.50:56444) 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 -- 172.21.15.50:0/3102741301 learned_addr learned my addr 172.21.15.50:0/3102741301 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d4be00700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 0x55ac641f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 -- 172.21.15.50:0/3102741301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 msgr2=0x55ac641f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 0x55ac641f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 -- 172.21.15.50:0/3102741301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac64064ec0 msgr2=0x55ac641f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:08.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac64064ec0 0x55ac641f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 -- 172.21.15.50:0/3102741301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac641f1c90 con 0x55ac641e88b0 2024-05-10T21:11:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d4be00700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 0x55ac641f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d51400700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009960 tx=0x7f5d44017ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5f9d47531f2f671 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.632+0000 7f5d32a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44023c90 con 0x55ac641e88b0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d380b3410 con 0x55ac641e88b0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5d44023440 con 0x55ac641e88b0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d4403e020 con 0x55ac641e88b0 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d61fc0b80 10 client.25898 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac642675e0 con 0x55ac641e88b0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.633+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d380b3410 con 0x55ac641e88b0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d44021020 con 0x55ac641e88b0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 20 client.25898 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac642679e0 0x55ac64269e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 4 client.25898 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac64112480 con 0x55ac642679e0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d50a00700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac642679e0 0x55ac64269e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d50a00700 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac642679e0 0x55ac64269e00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0af497e25496a6f server_cookie=9e7c8eb28d0946db in_seq=0 out_seq=0 2024-05-10T21:11:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.634+0000 7f5d32a00700 10 client.25898 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.635+0000 7f5d32a00700 1 -- 172.21.15.50:0/3102741301 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac64112480 con 0x55ac642679e0 2024-05-10T21:11:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.635+0000 7f5d32a00700 10 client.25898 handle_command_reply: tid=0 2024-05-10T21:11:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.635+0000 7f5d61fc0b80 1 client.25898 shutdown 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 msgr2=0x55ac641f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009960 tx=0x7f5d44017ed0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 shutdown_connections 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ac642679e0 0x55ac64269e00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ac641e88b0 0x55ac641f16d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac64064ec0 0x55ac641f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 --2- 172.21.15.50:0/3102741301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac641e2a70 0x55ac641f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 >> 172.21.15.50:0/3102741301 conn(0x55ac640a9ff0 msgr2=0x55ac641e64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d32a00700 0 client.25898 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 shutdown_connections 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 1 -- 172.21.15.50:0/3102741301 wait complete. 2024-05-10T21:11:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.636+0000 7f5d61fc0b80 20 client.25898 trim_cache size 0 max 16384 2024-05-10T21:11:08.637 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:11:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:11:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "531ef45e-ff23-459c-8638-150f1ebce62a", 2024-05-10T21:11:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:11:08.637 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.637+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 msgr2=0x7f5d340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.637+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 0x7f5d340b4d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d4401c000 tx=0x7f5d4401e000 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.637+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 msgr2=0x7f5d4c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.637+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d38005910 tx=0x7f5d3800b670 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 shutdown_connections 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f5d340b2880 0x7f5d340b4d90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c105480 0x7f5d4c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c10e8f0 0x7f5d4c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 --2- 172.21.15.50:0/907334012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d4c105dd0 0x7f5d4c19b3b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 >> 172.21.15.50:0/907334012 conn(0x7f5d4c0fef30 msgr2=0x7f5d4c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 shutdown_connections 2024-05-10T21:11:08.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:08.638+0000 7f5d53400700 1 -- 172.21.15.50:0/907334012 wait complete. 2024-05-10T21:11:08.903 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ANSI C header files... yes 2024-05-10T21:11:09.027 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-10T21:11:09.085 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:08 smithi050 ceph-mon[29546]: pgmap v528: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-10T21:11:09.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:08 smithi092 ceph-mon[34415]: pgmap v528: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-10T21:11:09.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:08 smithi142 ceph-mon[34846]: pgmap v528: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-10T21:11:09.187 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/types.h... yes 2024-05-10T21:11:09.320 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/stat.h... yes 2024-05-10T21:11:09.462 INFO:tasks.workunit.client.0.smithi050.stdout:checking for stdlib.h... yes 2024-05-10T21:11:09.563 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:11:09.587 INFO:tasks.workunit.client.0.smithi050.stdout:checking for string.h... yes 2024-05-10T21:11:09.698 INFO:tasks.workunit.client.0.smithi050.stdout:checking for memory.h... yes 2024-05-10T21:11:09.827 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strings.h... yes 2024-05-10T21:11:09.935 INFO:tasks.workunit.client.0.smithi050.stdout:checking for inttypes.h... yes 2024-05-10T21:11:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:10.048 INFO:tasks.workunit.client.0.smithi050.stdout:checking for stdint.h... yes 2024-05-10T21:11:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:09 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:11:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:09 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:09 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:09 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:09 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:11:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:09 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:11:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:09 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:09 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:09 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:09 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:11:10.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:09 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:11:10.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:09 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:09 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:10.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:09 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:10.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:09 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:11:10.165 INFO:tasks.workunit.client.0.smithi050.stdout:checking for unistd.h... yes 2024-05-10T21:11:10.194 INFO:tasks.workunit.client.0.smithi050.stdout:checking for memory.h... (cached) yes 2024-05-10T21:11:10.215 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strings.h... (cached) yes 2024-05-10T21:11:10.232 INFO:tasks.workunit.client.0.smithi050.stdout:checking for unistd.h... (cached) yes 2024-05-10T21:11:10.345 INFO:tasks.workunit.client.0.smithi050.stdout:checking locale.h usability... yes 2024-05-10T21:11:10.394 INFO:tasks.workunit.client.0.smithi050.stdout:checking locale.h presence... yes 2024-05-10T21:11:10.397 INFO:tasks.workunit.client.0.smithi050.stdout:checking for locale.h... yes 2024-05-10T21:11:10.432 INFO:tasks.workunit.client.0.smithi050.stdout:checking for stdlib.h... (cached) yes 2024-05-10T21:11:10.454 INFO:tasks.workunit.client.0.smithi050.stdout:checking for string.h... (cached) yes 2024-05-10T21:11:10.704 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/socket.h usability... yes 2024-05-10T21:11:10.753 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/socket.h presence... yes 2024-05-10T21:11:10.755 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/socket.h... yes 2024-05-10T21:11:10.895 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/time.h usability... yes 2024-05-10T21:11:10.949 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/time.h presence... yes 2024-05-10T21:11:10.952 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/time.h... yes 2024-05-10T21:11:11.081 INFO:tasks.workunit.client.0.smithi050.stdout:checking fcntl.h usability... yes 2024-05-10T21:11:11.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:10 smithi092 ceph-mon[34415]: pgmap v529: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-10T21:11:11.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:10 smithi142 ceph-mon[34846]: pgmap v529: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-10T21:11:11.143 INFO:tasks.workunit.client.0.smithi050.stdout:checking fcntl.h presence... yes 2024-05-10T21:11:11.253 INFO:tasks.workunit.client.0.smithi050.stdout:checking for fcntl.h... yes 2024-05-10T21:11:11.253 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:10 smithi050 ceph-mon[29546]: pgmap v529: 97 pgs: 97 active+clean; 854 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-10T21:11:11.292 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fcntl.h usability... yes 2024-05-10T21:11:11.352 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fcntl.h presence... yes 2024-05-10T21:11:11.355 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/fcntl.h... yes 2024-05-10T21:11:11.529 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/param.h usability... yes 2024-05-10T21:11:11.579 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/param.h presence... yes 2024-05-10T21:11:11.581 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/param.h... yes 2024-05-10T21:11:11.745 INFO:tasks.workunit.client.0.smithi050.stdout:checking ioctl.h usability... no 2024-05-10T21:11:11.809 INFO:tasks.workunit.client.0.smithi050.stdout:checking ioctl.h presence... no 2024-05-10T21:11:11.813 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ioctl.h... no 2024-05-10T21:11:11.948 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/ioctl.h usability... yes 2024-05-10T21:11:12.003 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/ioctl.h presence... yes 2024-05-10T21:11:12.009 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/ioctl.h... yes 2024-05-10T21:11:12.118 INFO:tasks.workunit.client.0.smithi050.stdout:checking return type of signal handlers... void 2024-05-10T21:11:12.217 INFO:tasks.workunit.client.0.smithi050.stdout:checking for size_t... yes 2024-05-10T21:11:12.315 INFO:tasks.workunit.client.0.smithi050.stdout:checking for dev_t... yes 2024-05-10T21:11:12.412 INFO:tasks.workunit.client.0.smithi050.stdout:checking for short... yes 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1780530479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa441034c0 msgr2=0x7ffa441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1780530479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa441034c0 0x7ffa441038d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098f0 tx=0x7ffa34009c00 comp rx=0 tx=0).stop 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1780530479 shutdown_connections 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1780530479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa441034c0 0x7ffa441038d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1780530479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44108740 0x7ffa441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1780530479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa44107df0 0x7ffa44108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.426+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1780530479 >> 172.21.15.50:0/1780530479 conn(0x7ffa440fedc0 msgr2=0x7ffa441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.427+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1780530479 shutdown_connections 2024-05-10T21:11:12.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.427+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1780530479 wait complete. 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.428+0000 7ffa4ac00700 1 Processor -- start 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.428+0000 7ffa4ac00700 1 -- start start 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.428+0000 7ffa4ac00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa4ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44107df0 0x7ffa440ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa4ac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa44108740 0x7ffa440ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa4ac00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441133c0 con 0x7ffa441034c0 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa4ac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44113240 con 0x7ffa44108740 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa4ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44113540 con 0x7ffa44107df0 2024-05-10T21:11:12.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa43e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa43e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34226/0 (socket says 172.21.15.50:34226) 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa43e00700 1 -- 172.21.15.50:0/1614180139 learned_addr learned my addr 172.21.15.50:0/1614180139 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa48c00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa44108740 0x7ffa440ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.429+0000 7ffa43400700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44107df0 0x7ffa440ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 -- 172.21.15.50:0/1614180139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44107df0 msgr2=0x7ffa440ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44107df0 0x7ffa440ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 -- 172.21.15.50:0/1614180139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa44108740 msgr2=0x7ffa440ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa44108740 0x7ffa440ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 -- 172.21.15.50:0/1614180139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441a9240 con 0x7ffa441034c0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa43e00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009230 tx=0x7ffa3400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa40c00700 1 -- 172.21.15.50:0/1614180139 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34005140 con 0x7ffa441034c0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.430+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441a94d0 con 0x7ffa441034c0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.431+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441a9980 con 0x7ffa441034c0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.431+0000 7ffa40c00700 1 -- 172.21.15.50:0/1614180139 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3400b560 con 0x7ffa441034c0 2024-05-10T21:11:12.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.431+0000 7ffa40c00700 1 -- 172.21.15.50:0/1614180139 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003ae0 con 0x7ffa441034c0 2024-05-10T21:11:12.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.435+0000 7ffa40c00700 1 -- 172.21.15.50:0/1614180139 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa34003c80 con 0x7ffa441034c0 2024-05-10T21:11:12.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.437+0000 7ffa59718b80 1 Processor -- start 2024-05-10T21:11:12.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.437+0000 7ffa40c00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 0x7ffa280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.437+0000 7ffa59718b80 1 -- start start 2024-05-10T21:11:12.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa40c00700 1 -- 172.21.15.50:0/1614180139 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffa340e3010 con 0x7ffa441034c0 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995748ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55599565ca20 con 0x555995749400 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5559955fa760 con 0x5559955bce40 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa59718b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559956589e0 con 0x555995743460 2024-05-10T21:11:12.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.438+0000 7ffa43400700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 0x7ffa280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa48c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa43400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa48c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45026/0 (socket says 172.21.15.50:45026) 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa48c00700 1 -- 172.21.15.50:0/3479880880 learned_addr learned my addr 172.21.15.50:0/3479880880 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995748ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.439+0000 7ffa43400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34240/0 (socket says 172.21.15.50:34240) 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 -- 172.21.15.50:0/3479880880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 msgr2=0x555995748ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995748ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 -- 172.21.15.50:0/3479880880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 msgr2=0x555995742f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 -- 172.21.15.50:0/3479880880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559955a98d0 con 0x555995743460 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa43400700 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa43400700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 0x7ffa280b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa441021e0 tx=0x7ffa300137e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.440+0000 7ffa48c00700 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0093a0 tx=0x7ffa3c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e2bf857b8e5dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.441+0000 7ffa39600700 1 -- 172.21.15.50:0/3479880880 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c003c00 con 0x555995743460 2024-05-10T21:11:12.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.441+0000 7ffa39600700 1 -- 172.21.15.50:0/3479880880 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3c003da0 con 0x555995743460 2024-05-10T21:11:12.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.441+0000 7ffa39600700 1 -- 172.21.15.50:0/3479880880 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c017960 con 0x555995743460 2024-05-10T21:11:12.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 -- 172.21.15.50:0/3479880880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 msgr2=0x555995743870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0093a0 tx=0x7ffa3c0099e0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 -- 172.21.15.50:0/3479880880 shutdown_connections 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995748ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x555995743870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3479880880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995742f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 -- 172.21.15.50:0/3479880880 >> 172.21.15.50:0/3479880880 conn(0x5559955fba80 msgr2=0x555995735790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 -- 172.21.15.50:0/3479880880 shutdown_connections 2024-05-10T21:11:12.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.442+0000 7ffa59718b80 1 -- 172.21.15.50:0/3479880880 wait complete. 2024-05-10T21:11:12.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.443+0000 7ffa59718b80 1 -- start start 2024-05-10T21:11:12.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.443+0000 7ffa59718b80 1 Processor -- start 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559955bce40 0x55599574cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995749400 0x55599574d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55599574fed0 con 0x5559955bce40 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5559955c2b20 con 0x555995749400 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555995750050 con 0x555995743460 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa59718b80 10 client.0 _ll_register_callbacks cb 0x5559955f0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:12.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa43400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa43400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45036/0 (socket says 172.21.15.50:45036) 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa43400700 1 -- 172.21.15.50:0/3098927584 learned_addr learned my addr 172.21.15.50:0/3098927584 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559955bce40 0x55599574cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.444+0000 7ffa48c00700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995749400 0x55599574d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 -- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995749400 msgr2=0x55599574d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995749400 0x55599574d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 -- 172.21.15.50:0/3098927584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559955bce40 msgr2=0x55599574cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559955bce40 0x55599574cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 -- 172.21.15.50:0/3098927584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555995622420 con 0x555995743460 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa43400700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000fbf0 tx=0x7ffa300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac90cae9f9ea87f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.445+0000 7ffa2e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30024070 con 0x555995743460 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffa30010c70 con 0x555995743460 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3c003da0 con 0x555995743460 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa3002f070 con 0x555995743460 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa59718b80 10 client.25876 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:12.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.446+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55599575aa40 con 0x555995743460 2024-05-10T21:11:12.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3c003da0 con 0x555995743460 2024-05-10T21:11:12.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ffa30003bf0 con 0x555995743460 2024-05-10T21:11:12.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:12.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 20 client.25876 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55599575d0d0 0x55599575f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 4 client.25876 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555995707070 con 0x55599575d0d0 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.447+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55599575d0d0 0x55599575f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.448+0000 7ffa2e000700 10 client.25876 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.448+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55599575d0d0 0x55599575f510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d75b824214444c server_cookie=a4a328df1516a387 in_seq=0 out_seq=0 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa2e000700 1 -- 172.21.15.50:0/3098927584 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555995707070 con 0x55599575d0d0 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa2e000700 10 client.25876 handle_command_reply: tid=0 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 client.25876 shutdown 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 msgr2=0x55599574d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000fbf0 tx=0x7ffa300047c0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 shutdown_connections 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55599575d0d0 0x55599575f510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995749400 0x55599574d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559955bce40 0x55599574cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3098927584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995743460 0x55599574d0d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 >> 172.21.15.50:0/3098927584 conn(0x5559955fba80 msgr2=0x555995741f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa2e000700 0 client.25876 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.449+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 shutdown_connections 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.450+0000 7ffa59718b80 1 -- 172.21.15.50:0/3098927584 wait complete. 2024-05-10T21:11:12.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.450+0000 7ffa59718b80 20 client.25876 trim_cache size 0 max 16384 2024-05-10T21:11:12.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 Processor -- start 2024-05-10T21:11:12.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 -- start start 2024-05-10T21:11:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995742f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995749400 0x555995758f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400fae0 con 0x555995743460 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400d370 con 0x5559955bce40 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.473+0000 7ffa59718b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400b1c0 con 0x555995749400 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34250/0 (socket says 172.21.15.50:34250) 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 -- 172.21.15.50:0/3221509012 learned_addr learned my addr 172.21.15.50:0/3221509012 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995742f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa48c00700 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995749400 0x555995758f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 -- 172.21.15.50:0/3221509012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995749400 msgr2=0x555995758f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995749400 0x555995758f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 -- 172.21.15.50:0/3221509012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 msgr2=0x555995742f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.474+0000 7ffa43400700 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995742f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.475+0000 7ffa43400700 1 -- 172.21.15.50:0/3221509012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599572efd0 con 0x555995743460 2024-05-10T21:11:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.475+0000 7ffa43400700 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009690 tx=0x7ffa300046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ae4e692a86ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.475+0000 7ffa38c00700 1 -- 172.21.15.50:0/3221509012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30024040 con 0x555995743460 2024-05-10T21:11:12.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.475+0000 7ffa38c00700 1 -- 172.21.15.50:0/3221509012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa30004440 con 0x555995743460 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.475+0000 7ffa38c00700 1 -- 172.21.15.50:0/3221509012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30003680 con 0x555995743460 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.476+0000 7ffa59718b80 1 -- 172.21.15.50:0/3221509012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 msgr2=0x5559957358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.476+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009690 tx=0x7ffa300046f0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 -- 172.21.15.50:0/3221509012 shutdown_connections 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5559955bce40 0x555995742f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555995749400 0x555995758f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3221509012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995743460 0x5559957358c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 -- 172.21.15.50:0/3221509012 >> 172.21.15.50:0/3221509012 conn(0x5559955fba80 msgr2=0x55599574bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 -- 172.21.15.50:0/3221509012 shutdown_connections 2024-05-10T21:11:12.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 -- 172.21.15.50:0/3221509012 wait complete. 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.477+0000 7ffa59718b80 1 -- start start 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 Processor -- start 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 0x5559957643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 0x5559957648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995764e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400b830 con 0x555995749400 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400cca0 con 0x555995743460 2024-05-10T21:11:12.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2400bf00 con 0x5559955bce40 2024-05-10T21:11:12.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.478+0000 7ffa59718b80 10 client.0 _ll_register_callbacks cb 0x5559955f0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:12.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa48c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995764e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 0x5559957643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa48c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995764e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34252/0 (socket says 172.21.15.50:34252) 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa48c00700 1 -- 172.21.15.50:0/3553665864 learned_addr learned my addr 172.21.15.50:0/3553665864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43400700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 0x5559957648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 -- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 msgr2=0x5559957648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 0x5559957648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 -- 172.21.15.50:0/3553665864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 msgr2=0x555995764e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995764e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.479+0000 7ffa43e00700 1 -- 172.21.15.50:0/3553665864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559957c6880 con 0x5559955bce40 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa43400700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 0x5559957648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa43e00700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 0x5559957643a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa340e0b40 tx=0x7ffa340b1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92b3427a23056b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa2e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340aa200 con 0x5559955bce40 2024-05-10T21:11:12.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7ffa340a6980 con 0x5559955bce40 2024-05-10T21:11:12.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa30004440 con 0x5559955bce40 2024-05-10T21:11:12.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa340aadf0 con 0x5559955bce40 2024-05-10T21:11:12.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa59718b80 10 client.25879 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:12.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559957c79c0 con 0x5559955bce40 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.480+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa30004440 con 0x5559955bce40 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ffa340aa950 con 0x5559955bce40 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 10 client.25879 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 20 client.25879 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5559957c8050 0x5559957ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 4 client.25879 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55599566a400 con 0x5559957c8050 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.481+0000 7ffa43400700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5559957c8050 0x5559957ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa43400700 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5559957c8050 0x5559957ca470 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8533061da0850b server_cookie=c471caa6839b219f in_seq=0 out_seq=0 2024-05-10T21:11:12.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa2e000700 10 client.25879 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa2e000700 1 -- 172.21.15.50:0/3553665864 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55599566a400 con 0x5559957c8050 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa2e000700 10 client.25879 handle_command_reply: tid=0 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa59718b80 1 client.25879 shutdown 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 msgr2=0x5559957643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 0x5559957643a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa340e0b40 tx=0x7ffa340b1040 comp rx=0 tx=0).stop 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 shutdown_connections 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.482+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5559957c8050 0x5559957ca470 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555995749400 0x555995764e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559955bce40 0x5559957643a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa2e000700 0 client.25879 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 --2- 172.21.15.50:0/3553665864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555995743460 0x5559957648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 >> 172.21.15.50:0/3553665864 conn(0x5559955fba80 msgr2=0x55599574bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 shutdown_connections 2024-05-10T21:11:12.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 1 -- 172.21.15.50:0/3553665864 wait complete. 2024-05-10T21:11:12.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.483+0000 7ffa59718b80 20 client.25879 trim_cache size 0 max 16384 2024-05-10T21:11:12.484 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:11:12.484 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:11:12.484 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:11:12.484 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 msgr2=0x7ffa280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 0x7ffa280b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa441021e0 tx=0x7ffa300137e0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 msgr2=0x7ffa440ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009230 tx=0x7ffa3400fa70 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 shutdown_connections 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2900 0x7ffa280b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa44108740 0x7ffa440ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa441034c0 0x7ffa440ff2a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 --2- 172.21.15.50:0/1614180139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa44107df0 0x7ffa440ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 >> 172.21.15.50:0/1614180139 conn(0x7ffa440fedc0 msgr2=0x7ffa441063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 shutdown_connections 2024-05-10T21:11:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:12.484+0000 7ffa4ac00700 1 -- 172.21.15.50:0/1614180139 wait complete. 2024-05-10T21:11:12.556 INFO:tasks.workunit.client.0.smithi050.stdout:checking size of short... 2 2024-05-10T21:11:12.656 INFO:tasks.workunit.client.0.smithi050.stdout:checking for int... yes 2024-05-10T21:11:12.789 INFO:tasks.workunit.client.0.smithi050.stdout:checking size of int... 4 2024-05-10T21:11:12.892 INFO:tasks.workunit.client.0.smithi050.stdout:checking for long... yes 2024-05-10T21:11:13.036 INFO:tasks.workunit.client.0.smithi050.stdout:checking size of long... 8 2024-05-10T21:11:13.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:12 smithi092 ceph-mon[34415]: pgmap v530: 97 pgs: 97 active+clean; 987 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-05-10T21:11:13.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:12 smithi142 ceph-mon[34846]: pgmap v530: 97 pgs: 97 active+clean; 987 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-05-10T21:11:13.136 INFO:tasks.workunit.client.0.smithi050.stdout:checking for long long... yes 2024-05-10T21:11:13.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:12 smithi050 ceph-mon[29546]: pgmap v530: 97 pgs: 97 active+clean; 987 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 8 op/s 2024-05-10T21:11:13.272 INFO:tasks.workunit.client.0.smithi050.stdout:checking size of long long... 8 2024-05-10T21:11:13.385 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ssize_t... yes 2024-05-10T21:11:13.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:11:13.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:11:13.484 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sig_atomic_t... yes 2024-05-10T21:11:13.570 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether gcc needs -traditional... no 2024-05-10T21:11:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 -- 172.21.15.50:0/2019714794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075990 msgr2=0x7f74dc10b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 --2- 172.21.15.50:0/2019714794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075990 0x7f74dc10b530 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009900 tx=0x7f74c802a410 comp rx=0 tx=0).stop 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 -- 172.21.15.50:0/2019714794 shutdown_connections 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 --2- 172.21.15.50:0/2019714794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075990 0x7f74dc10b530 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 --2- 172.21.15.50:0/2019714794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 --2- 172.21.15.50:0/2019714794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075040 0x7f74dc075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 -- 172.21.15.50:0/2019714794 >> 172.21.15.50:0/2019714794 conn(0x7f74dc0fec70 msgr2=0x7f74dc101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 -- 172.21.15.50:0/2019714794 shutdown_connections 2024-05-10T21:11:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.636+0000 7f74e2600640 1 -- 172.21.15.50:0/2019714794 wait complete. 2024-05-10T21:11:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 Processor -- start 2024-05-10T21:11:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 -- start start 2024-05-10T21:11:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 0x7f74dc19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075990 0x7f74dc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc1a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1117d0 con 0x7f74dc075990 2024-05-10T21:11:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc111650 con 0x7f74dc10ba70 2024-05-10T21:11:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.638+0000 7f74e2600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc111950 con 0x7f74dc075040 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 0x7f74dc19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075990 0x7f74dc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc1a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 0x7f74dc19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45054/0 (socket says 172.21.15.50:45054) 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1200640 1 -- 172.21.15.50:0/2472257316 learned_addr learned my addr 172.21.15.50:0/2472257316 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 -- 172.21.15.50:0/2472257316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 msgr2=0x7f74dc19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 0x7f74dc19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 -- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075990 msgr2=0x7f74dc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075990 0x7f74dc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.639+0000 7f74e1c00640 1 -- 172.21.15.50:0/2472257316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc1a3e80 con 0x7f74dc10ba70 2024-05-10T21:11:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74e1c00640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc1a3810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009fd0 tx=0x7f74d400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74d9600640 1 -- 172.21.15.50:0/2472257316 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d400ce20 con 0x7f74dc10ba70 2024-05-10T21:11:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74d9600640 1 -- 172.21.15.50:0/2472257316 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d4005020 con 0x7f74dc10ba70 2024-05-10T21:11:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc1a4170 con 0x7f74dc10ba70 2024-05-10T21:11:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74d9600640 1 -- 172.21.15.50:0/2472257316 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4010690 con 0x7f74dc10ba70 2024-05-10T21:11:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.640+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc1a4690 con 0x7f74dc10ba70 2024-05-10T21:11:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.644+0000 7f74eabc1740 1 Processor -- start 2024-05-10T21:11:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74d9600640 1 -- 172.21.15.50:0/2472257316 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f74d4010830 con 0x7f74dc10ba70 2024-05-10T21:11:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 -- start start 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d501b0 0x55be360f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be360f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be36056550 con 0x55be35d272f0 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55be35ffbbc0 con 0x55be35d2f4d0 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.646+0000 7f74eabc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be36052510 con 0x55be35d501b0 2024-05-10T21:11:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.647+0000 7f74e1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.647+0000 7f74e1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34284/0 (socket says 172.21.15.50:34284) 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.647+0000 7f74e1200640 1 -- 172.21.15.50:0/1275795586 learned_addr learned my addr 172.21.15.50:0/1275795586 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 -- 172.21.15.50:0/1275795586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d501b0 msgr2=0x55be360f2300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d501b0 0x55be360f2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 -- 172.21.15.50:0/1275795586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 msgr2=0x55be360f9780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be360f9780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 -- 172.21.15.50:0/1275795586 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be35d68210 con 0x55be35d272f0 2024-05-10T21:11:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74d9600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 0x7f74c40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.647 INFO:tasks.workunit.client.0.smithi050.stdout:checking for an ANSI C-conforming const... yes 2024-05-10T21:11:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74e1200640 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009d60 tx=0x7f74d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cacd1b1f9447b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74d9600640 1 -- 172.21.15.50:0/2472257316 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74d40e2800 con 0x7f74dc10ba70 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74ce000640 1 -- 172.21.15.50:0/1275795586 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0004220 con 0x55be35d272f0 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74ce000640 1 -- 172.21.15.50:0/1275795586 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00043c0 con 0x55be35d272f0 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.648+0000 7f74ce000640 1 -- 172.21.15.50:0/1275795586 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00035f0 con 0x55be35d272f0 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.649+0000 7f74dbe00640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 0x7f74c40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 -- 172.21.15.50:0/1275795586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 msgr2=0x55be360f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009d60 tx=0x7f74d0011810 comp rx=0 tx=0).stop 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 -- 172.21.15.50:0/1275795586 shutdown_connections 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be360f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d272f0 0x55be360f2880 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1275795586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d501b0 0x55be360f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 -- 172.21.15.50:0/1275795586 >> 172.21.15.50:0/1275795586 conn(0x55be35d44300 msgr2=0x55be360edc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74dbe00640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 0x7f74c40b4d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004770 tx=0x7f74c80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 -- 172.21.15.50:0/1275795586 shutdown_connections 2024-05-10T21:11:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.650+0000 7f74eabc1740 1 -- 172.21.15.50:0/1275795586 wait complete. 2024-05-10T21:11:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.651+0000 7f74eabc1740 1 -- start start 2024-05-10T21:11:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.651+0000 7f74eabc1740 1 Processor -- start 2024-05-10T21:11:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d2f4d0 0x55be360fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d501b0 0x55be36103240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be360fa1d0 con 0x55be35d2f4d0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55be3600f660 con 0x55be35d501b0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be360fa350 con 0x55be35d272f0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74eabc1740 10 client.0 _ll_register_callbacks cb 0x55be35fd9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.652+0000 7f74dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d2f4d0 0x55be360fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45078/0 (socket says 172.21.15.50:45078) 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 -- 172.21.15.50:0/1415741765 learned_addr learned my addr 172.21.15.50:0/1415741765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1c00640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d501b0 0x55be36103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 -- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d501b0 msgr2=0x55be36103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d501b0 0x55be36103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 -- 172.21.15.50:0/1415741765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d2f4d0 msgr2=0x55be360fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d2f4d0 0x55be360fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.653+0000 7f74e1200640 1 -- 172.21.15.50:0/1415741765 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be35e34620 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.654+0000 7f74e1200640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74d00057c0 tx=0x7f74d0019c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51f050ffa9211b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.654+0000 7f74c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.654+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0027030 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.654+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74d0019e00 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.654+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d00043c0 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.655+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74d001d040 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.655+0000 7f74eabc1740 10 client.25888 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.655+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be36106170 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.655+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00043c0 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f74d0005440 con 0x55be35d272f0 2024-05-10T21:11:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 10 client.25888 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 20 client.25888 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35d0ed80 0x55be361095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 4 client.25888 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be36064160 con 0x55be35d0ed80 2024-05-10T21:11:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.656+0000 7f74dbe00640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35d0ed80 0x55be361095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.657+0000 7f74dbe00640 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35d0ed80 0x55be361095d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800a9fed72ead063 server_cookie=5b25c471ac72830c in_seq=0 out_seq=0 2024-05-10T21:11:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.657+0000 7f74c2a00640 10 client.25888 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74c2a00640 1 -- 172.21.15.50:0/1415741765 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be36064160 con 0x55be35d0ed80 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74c2a00640 10 client.25888 handle_command_reply: tid=0 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 client.25888 shutdown 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 msgr2=0x55be360fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74d00057c0 tx=0x7f74d0019c10 comp rx=0 tx=0).stop 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 shutdown_connections 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35d0ed80 0x55be361095d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d501b0 0x55be36103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74c2a00640 0 client.25888 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d2f4d0 0x55be360fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1415741765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fbda0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.658+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 >> 172.21.15.50:0/1415741765 conn(0x55be35d44300 msgr2=0x55be360f0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.659+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 shutdown_connections 2024-05-10T21:11:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.659+0000 7f74eabc1740 1 -- 172.21.15.50:0/1415741765 wait complete. 2024-05-10T21:11:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.659+0000 7f74eabc1740 20 client.25888 trim_cache size 0 max 16384 2024-05-10T21:11:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 Processor -- start 2024-05-10T21:11:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 -- start start 2024-05-10T21:11:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be360ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1afbe0 con 0x55be35d501b0 2024-05-10T21:11:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1ad4b0 con 0x55be35d2f4d0 2024-05-10T21:11:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.694+0000 7f74eabc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1ab300 con 0x55be35d272f0 2024-05-10T21:11:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74e1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56540/0 (socket says 172.21.15.50:56540) 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 -- 172.21.15.50:0/2788953820 learned_addr learned my addr 172.21.15.50:0/2788953820 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74e1200640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45088/0 (socket says 172.21.15.50:45088) 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74e1c00640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be360ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 -- 172.21.15.50:0/2788953820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 msgr2=0x55be360fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 -- 172.21.15.50:0/2788953820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 msgr2=0x55be360ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be360ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 -- 172.21.15.50:0/2788953820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be360f95e0 con 0x55be35d2f4d0 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74e1c00640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be360ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.695+0000 7f74dbe00640 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009900 tx=0x7f74c803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f1cfebff4f9d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.696+0000 7f74cd600640 1 -- 172.21.15.50:0/2788953820 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8035050 con 0x55be35d2f4d0 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.696+0000 7f74cd600640 1 -- 172.21.15.50:0/2788953820 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8059070 con 0x55be35d2f4d0 2024-05-10T21:11:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.696+0000 7f74cd600640 1 -- 172.21.15.50:0/2788953820 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8003850 con 0x55be35d2f4d0 2024-05-10T21:11:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 -- 172.21.15.50:0/2788953820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 msgr2=0x55be36103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009900 tx=0x7f74c803e000 comp rx=0 tx=0).stop 2024-05-10T21:11:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 -- 172.21.15.50:0/2788953820 shutdown_connections 2024-05-10T21:11:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be360ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d2f4d0 0x55be36103160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 --2- 172.21.15.50:0/2788953820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d272f0 0x55be360fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 -- 172.21.15.50:0/2788953820 >> 172.21.15.50:0/2788953820 conn(0x55be35d44300 msgr2=0x55be360fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 -- 172.21.15.50:0/2788953820 shutdown_connections 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.697+0000 7f74eabc1740 1 -- 172.21.15.50:0/2788953820 wait complete. 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.698+0000 7f74eabc1740 1 -- start start 2024-05-10T21:11:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.698+0000 7f74eabc1740 1 Processor -- start 2024-05-10T21:11:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d2f4d0 0x55be361002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be3616f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1ab970 con 0x55be35d501b0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1acde0 con 0x55be35d272f0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1ac040 con 0x55be35d2f4d0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74eabc1740 10 client.0 _ll_register_callbacks cb 0x7f74d0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74e1200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74e1200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56556/0 (socket says 172.21.15.50:56556) 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74e1200640 1 -- 172.21.15.50:0/1510381365 learned_addr learned my addr 172.21.15.50:0/1510381365 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74dbe00640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d2f4d0 0x55be361002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.699+0000 7f74e1c00640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be3616f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 -- 172.21.15.50:0/1510381365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d2f4d0 msgr2=0x55be361002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d2f4d0 0x55be361002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 -- 172.21.15.50:0/1510381365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 msgr2=0x55be3616f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be3616f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 -- 172.21.15.50:0/1510381365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be36103260 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74e1200640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d0005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf8fbcbbed4faf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0027070 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f74d002e2d0 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.700+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c8059070 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74d00050d0 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 10 client.25931 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be36170cb0 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8059070 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f74d0003cd0 con 0x55be35d272f0 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 10 client.25931 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 20 client.25931 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35ce41c0 0x55be36173630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 4 client.25931 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be3605dfd0 con 0x55be35ce41c0 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.701+0000 7f74dbe00640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35ce41c0 0x55be36173630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.702+0000 7f74dbe00640 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35ce41c0 0x55be36173630 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b4e541659d2969 server_cookie=96cc94bac7411548 in_seq=0 out_seq=0 2024-05-10T21:11:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.702+0000 7f74c2a00640 10 client.25931 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:13.704 INFO:tasks.workunit.client.0.smithi050.stdout:checking for inline... inline 2024-05-10T21:11:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74c2a00640 1 -- 172.21.15.50:0/1510381365 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be3605dfd0 con 0x55be35ce41c0 2024-05-10T21:11:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74c2a00640 10 client.25931 handle_command_reply: tid=0 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 client.25931 shutdown 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 msgr2=0x55be360ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d0005c60 comp rx=0 tx=0).stop 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 shutdown_connections 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be35ce41c0 0x55be36173630 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.793+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be35d2f4d0 0x55be361002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be35d501b0 0x55be3616f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 1 --2- 172.21.15.50:0/1510381365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be35d272f0 0x55be360ffd60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74c2a00640 0 client.25931 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 >> 172.21.15.50:0/1510381365 conn(0x55be35d44300 msgr2=0x55be360f0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 shutdown_connections 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 1 -- 172.21.15.50:0/1510381365 wait complete. 2024-05-10T21:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.794+0000 7f74eabc1740 20 client.25931 trim_cache size 0 max 16384 2024-05-10T21:11:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:11:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:11:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:11:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 msgr2=0x7f74c40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 0x7f74c40b4d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004770 tx=0x7f74c80314a0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 msgr2=0x7f74dc1a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc1a3810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009fd0 tx=0x7f74d400ef10 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 shutdown_connections 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f74c40b2860 0x7f74c40b4d70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74dc10ba70 0x7f74dc1a3810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74dc075990 0x7f74dc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 --2- 172.21.15.50:0/2472257316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74dc075040 0x7f74dc19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 >> 172.21.15.50:0/2472257316 conn(0x7f74dc0fec70 msgr2=0x7f74dc0ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 shutdown_connections 2024-05-10T21:11:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:13.796+0000 7f74e2600640 1 -- 172.21.15.50:0/2472257316 wait complete. 2024-05-10T21:11:13.879 INFO:tasks.workunit.client.0.smithi050.stdout:checking for working alloca.h... yes 2024-05-10T21:11:13.960 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:13.984 INFO:tasks.workunit.client.0.smithi050.stdout:checking for alloca... yes 2024-05-10T21:11:14.134 INFO:tasks.workunit.client.0.smithi050.stdout:checking for working memcmp... yes 2024-05-10T21:11:14.259 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strftime... yes 2024-05-10T21:11:14.406 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-05-10T21:11:14.529 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether stat accepts an empty string... no 2024-05-10T21:11:14.655 INFO:tasks.workunit.client.0.smithi050.stdout:checking for vprintf... yes 2024-05-10T21:11:14.777 INFO:tasks.workunit.client.0.smithi050.stdout:checking for _doprnt... no 2024-05-10T21:11:14.848 INFO:tasks.workunit.client.0.smithi050.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-10T21:11:14.886 INFO:tasks.workunit.client.0.smithi050.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-10T21:11:14.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:14 smithi050 ceph-mon[29546]: pgmap v531: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 165 KiB/s wr, 17 op/s 2024-05-10T21:11:15.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:14 smithi092 ceph-mon[34415]: pgmap v531: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 165 KiB/s wr, 17 op/s 2024-05-10T21:11:15.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:14 smithi142 ceph-mon[34846]: pgmap v531: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 165 KiB/s wr, 17 op/s 2024-05-10T21:11:15.135 INFO:tasks.workunit.client.0.smithi050.stdout:checking for the pthreads library -lpthreads... no 2024-05-10T21:11:15.271 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether pthreads work without any flags... yes 2024-05-10T21:11:15.396 INFO:tasks.workunit.client.0.smithi050.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-05-10T21:11:15.396 INFO:tasks.workunit.client.0.smithi050.stdout:checking if more special flags are required for pthreads... no 2024-05-10T21:11:15.396 INFO:tasks.workunit.client.0.smithi050.stdout:checking for cc_r... gcc 2024-05-10T21:11:15.565 INFO:tasks.workunit.client.0.smithi050.stdout:checking for setrlimit... yes 2024-05-10T21:11:15.723 INFO:tasks.workunit.client.0.smithi050.stdout:checking for setproctitle... no 2024-05-10T21:11:15.846 INFO:tasks.workunit.client.0.smithi050.stdout:checking for getopt_long... yes 2024-05-10T21:11:15.994 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strtoull... yes 2024-05-10T21:11:16.152 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strtoq... yes 2024-05-10T21:11:16.299 INFO:tasks.workunit.client.0.smithi050.stdout:checking for snprintf... yes 2024-05-10T21:11:16.446 INFO:tasks.workunit.client.0.smithi050.stdout:checking for vsnprintf... yes 2024-05-10T21:11:16.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 -- 172.21.15.50:0/3284845504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 msgr2=0x7fb91810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:16.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 --2- 172.21.15.50:0/3284845504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb91810c9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b0a0 tx=0x7fb90c00b3b0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 -- 172.21.15.50:0/3284845504 shutdown_connections 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 --2- 172.21.15.50:0/3284845504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106ed0 0x7fb918113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 --2- 172.21.15.50:0/3284845504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb91810c9d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 --2- 172.21.15.50:0/3284845504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106580 0x7fb918106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 -- 172.21.15.50:0/3284845504 >> 172.21.15.50:0/3284845504 conn(0x7fb918102080 msgr2=0x7fb9181044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 -- 172.21.15.50:0/3284845504 shutdown_connections 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.492+0000 7fb91f400700 1 -- 172.21.15.50:0/3284845504 wait complete. 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.494+0000 7fb91f400700 1 Processor -- start 2024-05-10T21:11:16.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.494+0000 7fb91f400700 1 -- start start 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 0x7fb91819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb9181a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb918116720 con 0x7fb91810c5c0 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181165a0 con 0x7fb918106ed0 2024-05-10T21:11:16.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91f400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181168a0 con 0x7fb918106580 2024-05-10T21:11:16.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:16.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 0x7fb91819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37828/0 (socket says 172.21.15.50:37828) 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 -- 172.21.15.50:0/785944076 learned_addr learned my addr 172.21.15.50:0/785944076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb91d400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb9181a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 -- 172.21.15.50:0/785944076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 msgr2=0x7fb91819d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 0x7fb91819d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 -- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 msgr2=0x7fb9181a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb9181a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.495+0000 7fb917e00700 1 -- 172.21.15.50:0/785944076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9181a1fe0 con 0x7fb918106ed0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb91d400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb9181a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb91ca00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 0x7fb91819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb917e00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb904009fd0 tx=0x7fb90400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90400cbd0 con 0x7fb918106ed0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9040044d0 con 0x7fb918106ed0 2024-05-10T21:11:16.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a22d0 con 0x7fb918106ed0 2024-05-10T21:11:16.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904003e50 con 0x7fb918106ed0 2024-05-10T21:11:16.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.496+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a9bf0 con 0x7fb918106ed0 2024-05-10T21:11:16.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.497+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91810e980 con 0x7fb918106ed0 2024-05-10T21:11:16.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.500+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb904004000 con 0x7fb918106ed0 2024-05-10T21:11:16.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.501+0000 7fb915600700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 0x7fb9000b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:16.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.502+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fb9040e2470 con 0x7fb918106ed0 2024-05-10T21:11:16.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.502+0000 7fb91ca00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 0x7fb9000b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:16.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.503+0000 7fb91ca00700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 0x7fb9000b4fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b780 tx=0x7fb90c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:16.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.506+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9040042f0 con 0x7fb918106ed0 2024-05-10T21:11:16.584 INFO:tasks.workunit.client.0.smithi050.stdout:checking for vfprintf... yes 2024-05-10T21:11:16.720 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gethostname... yes 2024-05-10T21:11:16.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.801+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb91819e470 con 0x7fb918106ed0 2024-05-10T21:11:16.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.802+0000 7fb915600700 1 -- 172.21.15.50:0/785944076 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fb9040aa160 con 0x7fb918106ed0 2024-05-10T21:11:16.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:11:16.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:08:17.695422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1752993045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1752993045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1752993045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.807+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 msgr2=0x7fb9000b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.807+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 0x7fb9000b4fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b780 tx=0x7fb90c006210 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.807+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 msgr2=0x7fb91819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.807+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb904009fd0 tx=0x7fb90400ed10 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 shutdown_connections 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb9000b2aa0 0x7fb9000b4fb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb91810c5c0 0x7fb9181a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb918106580 0x7fb91819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 --2- 172.21.15.50:0/785944076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb918106ed0 0x7fb91819d6b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:16.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 >> 172.21.15.50:0/785944076 conn(0x7fb918102080 msgr2=0x7fb9181048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:16.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 shutdown_connections 2024-05-10T21:11:16.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:16.808+0000 7fb91f400700 1 -- 172.21.15.50:0/785944076 wait complete. 2024-05-10T21:11:16.809 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-05-10T21:11:16.870 INFO:tasks.workunit.client.0.smithi050.stdout:checking for setlocale... yes 2024-05-10T21:11:17.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:16 smithi092 ceph-mon[34415]: pgmap v532: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 188 KiB/s wr, 43 op/s 2024-05-10T21:11:17.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:16 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/785944076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:16 smithi142 ceph-mon[34846]: pgmap v532: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 188 KiB/s wr, 43 op/s 2024-05-10T21:11:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:16 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/785944076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:17.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:16 smithi050 ceph-mon[29546]: pgmap v532: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 188 KiB/s wr, 43 op/s 2024-05-10T21:11:17.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:16 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/785944076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:11:17.331 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sync... yes 2024-05-10T21:11:18.060 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether snprintf is C99 conformant... done 2024-05-10T21:11:18.093 INFO:tasks.workunit.client.0.smithi050.stdout:configure: creating ./config.status 2024-05-10T21:11:18.515 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating Makefile 2024-05-10T21:11:18.568 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating src/Makefile 2024-05-10T21:11:18.624 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating man/Makefile 2024-05-10T21:11:18.650 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating config.h 2024-05-10T21:11:18.790 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: executing depfiles commands 2024-05-10T21:11:18.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:18 smithi050 ceph-mon[29546]: pgmap v533: 97 pgs: 97 active+clean; 3.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 306 KiB/s wr, 82 op/s 2024-05-10T21:11:19.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:18 smithi092 ceph-mon[34415]: pgmap v533: 97 pgs: 97 active+clean; 3.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 306 KiB/s wr, 82 op/s 2024-05-10T21:11:19.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:18 smithi142 ceph-mon[34846]: pgmap v533: 97 pgs: 97 active+clean; 3.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 306 KiB/s wr, 82 op/s 2024-05-10T21:11:19.215 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:11:19.226 INFO:tasks.workunit.client.0.smithi050.stderr:+ make 2024-05-10T21:11:19.242 INFO:tasks.workunit.client.0.smithi050.stdout:make all-recursive 2024-05-10T21:11:19.246 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-10T21:11:19.254 INFO:tasks.workunit.client.0.smithi050.stdout:Making all in src 2024-05-10T21:11:19.262 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-10T21:11:19.262 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-05-10T21:11:19.262 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-05-10T21:11:19.516 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-05-10T21:11:19.516 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-05-10T21:11:19.679 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:19.692 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-05-10T21:11:19.693 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-05-10T21:11:19.839 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-05-10T21:11:19.839 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-05-10T21:11:20.006 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-05-10T21:11:20.007 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-05-10T21:11:20.180 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-05-10T21:11:20.180 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-05-10T21:11:20.314 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-05-10T21:11:20.314 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-05-10T21:11:20.520 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-05-10T21:11:20.520 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-05-10T21:11:20.679 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-05-10T21:11:20.679 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-05-10T21:11:20.820 INFO:tasks.workunit.client.0.smithi050.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-05-10T21:11:20.820 INFO:tasks.workunit.client.0.smithi050.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-05-10T21:11:20.951 INFO:tasks.workunit.client.0.smithi050.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-05-10T21:11:20.990 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-05-10T21:11:20.991 INFO:tasks.workunit.client.0.smithi050.stdout:Making all in man 2024-05-10T21:11:20.999 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-10T21:11:20.999 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Nothing to be done for 'all'. 2024-05-10T21:11:20.999 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-05-10T21:11:21.004 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-10T21:11:21.008 INFO:tasks.workunit.client.0.smithi050.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-10T21:11:21.009 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-05-10T21:11:21.011 INFO:tasks.workunit.client.0.smithi050.stderr:+ cd src 2024-05-10T21:11:21.015 INFO:tasks.workunit.client.0.smithi050.stderr:+ mkdir blogtest_in 2024-05-10T21:11:21.022 INFO:tasks.workunit.client.0.smithi050.stdout:running blogbench 2024-05-10T21:11:21.023 INFO:tasks.workunit.client.0.smithi050.stderr:+ echo 'running blogbench' 2024-05-10T21:11:21.023 INFO:tasks.workunit.client.0.smithi050.stderr:+ ./blogbench -d blogtest_in 2024-05-10T21:11:21.025 INFO:tasks.workunit.client.0.smithi050.stdout: 2024-05-10T21:11:21.025 INFO:tasks.workunit.client.0.smithi050.stdout:Frequency = 10 secs 2024-05-10T21:11:21.025 INFO:tasks.workunit.client.0.smithi050.stdout:Scratch dir = [blogtest_in] 2024-05-10T21:11:21.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:20 smithi092 ceph-mon[34415]: pgmap v534: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 321 KiB/s wr, 93 op/s 2024-05-10T21:11:21.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:20 smithi142 ceph-mon[34846]: pgmap v534: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 321 KiB/s wr, 93 op/s 2024-05-10T21:11:21.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:20 smithi050 ceph-mon[29546]: pgmap v534: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 321 KiB/s wr, 93 op/s 2024-05-10T21:11:21.366 INFO:tasks.workunit.client.0.smithi050.stdout:Spawning 3 writers... 2024-05-10T21:11:21.366 INFO:tasks.workunit.client.0.smithi050.stdout:Spawning 1 rewriters... 2024-05-10T21:11:21.366 INFO:tasks.workunit.client.0.smithi050.stdout:Spawning 5 commenters... 2024-05-10T21:11:21.367 INFO:tasks.workunit.client.0.smithi050.stdout:Spawning 100 readers... 2024-05-10T21:11:21.374 INFO:tasks.workunit.client.0.smithi050.stdout:Benchmarking for 30 iterations. 2024-05-10T21:11:21.374 INFO:tasks.workunit.client.0.smithi050.stdout:The test will run during 5 minutes. 2024-05-10T21:11:21.375 INFO:tasks.workunit.client.0.smithi050.stdout: 2024-05-10T21:11:21.375 INFO:tasks.workunit.client.0.smithi050.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-05-10T21:11:22.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 -- 172.21.15.50:0/1190479734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 msgr2=0x7f97f0110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 --2- 172.21.15.50:0/1190479734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0110fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b0a0 tx=0x7f97e800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 -- 172.21.15.50:0/1190479734 shutdown_connections 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 --2- 172.21.15.50:0/1190479734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0110fe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 --2- 172.21.15.50:0/1190479734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 0x7f97f0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 --2- 172.21.15.50:0/1190479734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 -- 172.21.15.50:0/1190479734 >> 172.21.15.50:0/1190479734 conn(0x7f97f0079d10 msgr2=0x7f97f007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 -- 172.21.15.50:0/1190479734 shutdown_connections 2024-05-10T21:11:22.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.530+0000 7f97ef200700 1 -- 172.21.15.50:0/1190479734 wait complete. 2024-05-10T21:11:22.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.531+0000 7f97ef200700 1 Processor -- start 2024-05-10T21:11:22.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 -- start start 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 0x7f97f0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0114300 con 0x7f97f0108d30 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0114180 con 0x7f97f0109680 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ef200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0114480 con 0x7f97f0104400 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ee800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ee800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37836/0 (socket says 172.21.15.50:37836) 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ee800700 1 -- 172.21.15.50:0/2615451378 learned_addr learned my addr 172.21.15.50:0/2615451378 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.532+0000 7f97ed400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ede00700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 0x7f97f0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 -- 172.21.15.50:0/2615451378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 msgr2=0x7f97f0075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 0x7f97f0075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 -- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 msgr2=0x7f97f0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 -- 172.21.15.50:0/2615451378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f01a8fd0 con 0x7f97f0109680 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ed400700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.533+0000 7f97ee800700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b070 tx=0x7f97e8004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.534+0000 7f97e6a00700 1 -- 172.21.15.50:0/2615451378 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8021070 con 0x7f97f0109680 2024-05-10T21:11:22.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.534+0000 7f97e6a00700 1 -- 172.21.15.50:0/2615451378 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e8003c00 con 0x7f97f0109680 2024-05-10T21:11:22.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.534+0000 7f97e6a00700 1 -- 172.21.15.50:0/2615451378 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8004700 con 0x7f97f0109680 2024-05-10T21:11:22.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.534+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f01a9260 con 0x7f97f0109680 2024-05-10T21:11:22.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.534+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f01a96c0 con 0x7f97f0109680 2024-05-10T21:11:22.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.540+0000 7f97e6a00700 1 -- 172.21.15.50:0/2615451378 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f97e8007980 con 0x7f97f0109680 2024-05-10T21:11:22.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 Processor -- start 2024-05-10T21:11:22.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 -- start start 2024-05-10T21:11:22.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf2423e280 0x55bf242478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24249e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf2415fa20 con 0x55bf2423e280 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf240fd760 con 0x55bf240bfe40 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.541+0000 7f9803d83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf2415b9e0 con 0x55bf2423e690 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ede00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf2423e280 0x55bf242478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37838/0 (socket says 172.21.15.50:37838) 2024-05-10T21:11:22.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 -- 172.21.15.50:0/3725075007 learned_addr learned my addr 172.21.15.50:0/3725075007 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:22.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97e6a00700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 0x7f97d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24249e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97e6a00700 1 -- 172.21.15.50:0/2615451378 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f97e800f020 con 0x7f97f0109680 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 -- 172.21.15.50:0/3725075007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 msgr2=0x55bf24249e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24249e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 -- 172.21.15.50:0/3725075007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf2423e280 msgr2=0x55bf242478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf2423e280 0x55bf242478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 -- 172.21.15.50:0/3725075007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf240ac8d0 con 0x55bf240bfe40 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ee800700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 0x7f97d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.542+0000 7f97ed400700 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009ee0 tx=0x7f97dc009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11220ed96174fe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.543+0000 7f97db400700 1 -- 172.21.15.50:0/3725075007 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc012e40 con 0x55bf240bfe40 2024-05-10T21:11:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.543+0000 7f97db400700 1 -- 172.21.15.50:0/3725075007 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97dc0044d0 con 0x55bf240bfe40 2024-05-10T21:11:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.543+0000 7f97db400700 1 -- 172.21.15.50:0/3725075007 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc010420 con 0x55bf240bfe40 2024-05-10T21:11:22.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.543+0000 7f97ee800700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 0x7f97d40b4e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97e80aa9e0 tx=0x7f97e8009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.544+0000 7f9803d83b80 1 -- 172.21.15.50:0/3725075007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 msgr2=0x55bf2423dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.544+0000 7f9803d83b80 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009ee0 tx=0x7f97dc009750 comp rx=0 tx=0).stop 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 -- 172.21.15.50:0/3725075007 shutdown_connections 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf240bfe40 0x55bf2423dd40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24249e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 --2- 172.21.15.50:0/3725075007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf2423e280 0x55bf242478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 -- 172.21.15.50:0/3725075007 >> 172.21.15.50:0/3725075007 conn(0x55bf240fea80 msgr2=0x55bf24238b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 -- 172.21.15.50:0/3725075007 shutdown_connections 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 -- 172.21.15.50:0/3725075007 wait complete. 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.545+0000 7f9803d83b80 1 -- start start 2024-05-10T21:11:22.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 Processor -- start 2024-05-10T21:11:22.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf24247400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24247e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf2424a7a0 con 0x55bf240bfe40 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf240c5b20 con 0x55bf2423e280 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf2424a920 con 0x55bf2423e690 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.546+0000 7f9803d83b80 10 client.0 _ll_register_callbacks cb 0x55bf240f3380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ede00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf24247400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ede00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37848/0 (socket says 172.21.15.50:37848) 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ede00700 1 -- 172.21.15.50:0/1664762616 learned_addr learned my addr 172.21.15.50:0/1664762616 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24247e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 -- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 msgr2=0x55bf24247e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24247e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 -- 172.21.15.50:0/1664762616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 msgr2=0x55bf24247940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 -- 172.21.15.50:0/1664762616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf24125420 con 0x55bf240bfe40 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ede00700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.547+0000 7f97ed400700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf24247400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0046a0 tx=0x7f97dc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14046bd18c73b53e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f97cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc003880 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f97dc005030 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97dc0044d0 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97dc005230 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f9803d83b80 10 client.15822 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.548+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf242554a0 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97dc0044d0 con 0x55bf240bfe40 2024-05-10T21:11:22.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97dc01e440 con 0x55bf240bfe40 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 20 client.15822 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf24257970 0x55bf24259d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 4 client.15822 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.549+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf2420a070 con 0x55bf24257970 2024-05-10T21:11:22.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.550+0000 7f97ede00700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf24257970 0x55bf24259d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.550+0000 7f97ede00700 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf24257970 0x55bf24259d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39915e5e07501f85 server_cookie=e7bacdc1934d3e44 in_seq=0 out_seq=0 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.550+0000 7f97cfe00700 10 client.15822 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.551+0000 7f97cfe00700 1 -- 172.21.15.50:0/1664762616 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf2420a070 con 0x55bf24257970 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.551+0000 7f97cfe00700 10 client.15822 handle_command_reply: tid=0 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.551+0000 7f9803d83b80 1 client.15822 shutdown 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 msgr2=0x55bf24247400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf24247400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0046a0 tx=0x7f97dc009bf0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 shutdown_connections 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf24257970 0x55bf24259d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e690 0x55bf24247e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf24247400 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1664762616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e280 0x55bf24247940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 >> 172.21.15.50:0/1664762616 conn(0x55bf240fea80 msgr2=0x55bf2423c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f97cfe00700 0 client.15822 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 shutdown_connections 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 1 -- 172.21.15.50:0/1664762616 wait complete. 2024-05-10T21:11:22.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.552+0000 7f9803d83b80 20 client.15822 trim_cache size 0 max 16384 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 Processor -- start 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 -- start start 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf242465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.570+0000 7f9803d83b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01b32a0 con 0x55bf240bfe40 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f9803d83b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01b0b30 con 0x55bf2423e690 2024-05-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f9803d83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01ae9e0 con 0x55bf2423e280 2024-05-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ed400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf242465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ed400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf242465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47086/0 (socket says 172.21.15.50:47086) 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ed400700 1 -- 172.21.15.50:0/2315998335 learned_addr learned my addr 172.21.15.50:0/2315998335 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60344/0 (socket says 172.21.15.50:60344) 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ed400700 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 -- 172.21.15.50:0/2315998335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 msgr2=0x55bf242491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 -- 172.21.15.50:0/2315998335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 msgr2=0x55bf242465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf242465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ede00700 1 -- 172.21.15.50:0/2315998335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2423eaa0 con 0x55bf2423e280 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.571+0000 7f97ed400700 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.572+0000 7f97ede00700 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97e00097a0 tx=0x7f97e00125f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369ca739f2dbc015 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.572+0000 7f97daa00700 1 -- 172.21.15.50:0/2315998335 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0027070 con 0x55bf2423e280 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.572+0000 7f97daa00700 1 -- 172.21.15.50:0/2315998335 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e0003680 con 0x55bf2423e280 2024-05-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.572+0000 7f97daa00700 1 -- 172.21.15.50:0/2315998335 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e00215a0 con 0x55bf2423e280 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.573+0000 7f9803d83b80 1 -- 172.21.15.50:0/2315998335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 msgr2=0x55bf24247cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.573+0000 7f9803d83b80 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97e00097a0 tx=0x7f97e00125f0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 -- 172.21.15.50:0/2315998335 shutdown_connections 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf242465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 --2- 172.21.15.50:0/2315998335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf24247cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 -- 172.21.15.50:0/2315998335 >> 172.21.15.50:0/2315998335 conn(0x55bf240fea80 msgr2=0x55bf24248ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 -- 172.21.15.50:0/2315998335 shutdown_connections 2024-05-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.574+0000 7f9803d83b80 1 -- 172.21.15.50:0/2315998335 wait complete. 2024-05-10T21:11:22.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 -- start start 2024-05-10T21:11:22.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 Processor -- start 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf2424ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01af030 con 0x55bf240bfe40 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01b0460 con 0x55bf2423e690 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.575+0000 7f9803d83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97f01af6c0 con 0x55bf2423e280 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f9803d83b80 10 client.0 _ll_register_callbacks cb 0x55bf24254220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ed400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf2424ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60352/0 (socket says 172.21.15.50:60352) 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 -- 172.21.15.50:0/1833992987 learned_addr learned my addr 172.21.15.50:0/1833992987 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ed400700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 -- 172.21.15.50:0/1833992987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 msgr2=0x55bf242bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 -- 172.21.15.50:0/1833992987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 msgr2=0x55bf2424ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf2424ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ede00700 1 -- 172.21.15.50:0/1833992987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf242c0470 con 0x55bf2423e280 2024-05-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.576+0000 7f97ed400700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf2424ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:22.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.577+0000 7f97ede00700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f97e00212e0 tx=0x7f97e0021310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c22155b52c8bfee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:22.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.577+0000 7f97cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.577+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0027040 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.577+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e0003680 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.577+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f97e0021730 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.578+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97e0021970 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.578+0000 7f9803d83b80 10 client.25918 fetch_fsmap learned FSMap version 25 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.578+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf242c15f0 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.578+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e0003680 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97e002a020 con 0x55bf2423e280 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 10 client.25918 fetch_fsmap finished waiting for FSMap version 25 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 10 client.25918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:11:22.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 20 client.25918 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:22.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf242c1c80 0x55bf242c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:22.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 4 client.25918 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:11:22.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf2416d400 con 0x55bf242c1c80 2024-05-10T21:11:22.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.579+0000 7f97ed400700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf242c1c80 0x55bf242c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f97ed400700 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf242c1c80 0x55bf242c40a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16daba09886cfc57 server_cookie=17f3c202a8bdf817 in_seq=0 out_seq=0 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f97cfe00700 10 client.25918 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f97cfe00700 1 -- 172.21.15.50:0/1833992987 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf2416d400 con 0x55bf242c1c80 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f97cfe00700 10 client.25918 handle_command_reply: tid=0 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 client.25918 shutdown 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 msgr2=0x55bf2424b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f97e00212e0 tx=0x7f97e0021310 comp rx=0 tx=0).stop 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 shutdown_connections 2024-05-10T21:11:22.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55bf242c1c80 0x55bf242c40a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf2423e690 0x55bf242bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf240bfe40 0x55bf2424ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 --2- 172.21.15.50:0/1833992987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf2423e280 0x55bf2424b3a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 >> 172.21.15.50:0/1833992987 conn(0x55bf240fea80 msgr2=0x55bf24246350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.580+0000 7f97cfe00700 0 client.25918 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.581+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 shutdown_connections 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.581+0000 7f9803d83b80 1 -- 172.21.15.50:0/1833992987 wait complete. 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.581+0000 7f9803d83b80 20 client.25918 trim_cache size 0 max 16384 2024-05-10T21:11:22.581 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 msgr2=0x7f97d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 0x7f97d40b4e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97e80aa9e0 tx=0x7f97e8009c10 comp rx=0 tx=0).stop 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 msgr2=0x7f97f0076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b070 tx=0x7f97e8004c60 comp rx=0 tx=0).stop 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 shutdown_connections 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f97d40b2900 0x7f97d40b4e10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109680 0x7f97f0076540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f0104400 0x7f97f0075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 --2- 172.21.15.50:0/2615451378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97f0108d30 0x7f97f0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:22.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 >> 172.21.15.50:0/2615451378 conn(0x7f97f0079d10 msgr2=0x7f97f01013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:22.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 shutdown_connections 2024-05-10T21:11:22.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:22.582+0000 7f97ef200700 1 -- 172.21.15.50:0/2615451378 wait complete. 2024-05-10T21:11:23.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:22 smithi092 ceph-mon[34415]: pgmap v535: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 330 KiB/s wr, 97 op/s 2024-05-10T21:11:23.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:22 smithi142 ceph-mon[34846]: pgmap v535: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 330 KiB/s wr, 97 op/s 2024-05-10T21:11:23.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:22 smithi050 ceph-mon[29546]: pgmap v535: 97 pgs: 97 active+clean; 4.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 330 KiB/s wr, 97 op/s 2024-05-10T21:11:23.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:11:24.200 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:25.106 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:24 smithi050 ceph-mon[29546]: pgmap v536: 97 pgs: 97 active+clean; 5.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 471 KiB/s wr, 108 op/s 2024-05-10T21:11:25.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:24 smithi142 ceph-mon[34846]: pgmap v536: 97 pgs: 97 active+clean; 5.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 471 KiB/s wr, 108 op/s 2024-05-10T21:11:25.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:24 smithi092 ceph-mon[34415]: pgmap v536: 97 pgs: 97 active+clean; 5.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 471 KiB/s wr, 108 op/s 2024-05-10T21:11:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:25 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:26.268 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:25 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:26.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:25 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 -- 172.21.15.50:0/2132461838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105e50 msgr2=0x7f576810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 --2- 172.21.15.50:0/2132461838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105e50 0x7f576810e3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57500098f0 tx=0x7f5750009c00 comp rx=0 tx=0).stop 2024-05-10T21:11:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 -- 172.21.15.50:0/2132461838 shutdown_connections 2024-05-10T21:11:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 --2- 172.21.15.50:0/2132461838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f576810e8e0 0x7f5768110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 --2- 172.21.15.50:0/2132461838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5768105500 0x7f5768105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 --2- 172.21.15.50:0/2132461838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105e50 0x7f576810e3a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.455+0000 7f576d600700 1 -- 172.21.15.50:0/2132461838 >> 172.21.15.50:0/2132461838 conn(0x7f57680fedc0 msgr2=0x7f57681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.456+0000 7f576d600700 1 -- 172.21.15.50:0/2132461838 shutdown_connections 2024-05-10T21:11:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.456+0000 7f576d600700 1 -- 172.21.15.50:0/2132461838 wait complete. 2024-05-10T21:11:26.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.458+0000 7f576d600700 1 Processor -- start 2024-05-10T21:11:26.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.458+0000 7f576d600700 1 -- start start 2024-05-10T21:11:26.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 0x7f57681a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 0x7f57681a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 0x7f57681ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5768114490 con 0x7f576810e8e0 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5768114310 con 0x7f5768105500 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.459+0000 7f576d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5768114610 con 0x7f5768105e50 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.460+0000 7f5766a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 0x7f57681a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.460+0000 7f5766a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 0x7f57681a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43114/0 (socket says 172.21.15.50:43114) 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.460+0000 7f5766a00700 1 -- 172.21.15.50:0/1844568646 learned_addr learned my addr 172.21.15.50:0/1844568646 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:26.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.460+0000 7f5766000700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 0x7f57681a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.461+0000 7f5766000700 1 -- 172.21.15.50:0/1844568646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 msgr2=0x7f57681a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.461+0000 7f5766000700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 0x7f57681a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.461+0000 7f5766000700 1 -- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 msgr2=0x7f57681ad1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T21:11:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.462+0000 7f5767400700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 0x7f57681ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.462+0000 7f5766000700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 0x7f57681ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.462+0000 7f5766000700 1 -- 172.21.15.50:0/1844568646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5768199960 con 0x7f5768105e50 2024-05-10T21:11:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.462+0000 7f5767400700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 0x7f57681ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.463+0000 7f5766000700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 0x7f57681a26b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f575000c010 tx=0x7f575000fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.464+0000 7f575f400700 1 -- 172.21.15.50:0/1844568646 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750005270 con 0x7f5768105e50 2024-05-10T21:11:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.465+0000 7f575f400700 1 -- 172.21.15.50:0/1844568646 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575000ba10 con 0x7f5768105e50 2024-05-10T21:11:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.465+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5768199bf0 con 0x7f5768105e50 2024-05-10T21:11:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.465+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576819a0d0 con 0x7f5768105e50 2024-05-10T21:11:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.465+0000 7f575f400700 1 -- 172.21.15.50:0/1844568646 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750003b40 con 0x7f5768105e50 2024-05-10T21:11:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.475+0000 7f577c208b80 1 Processor -- start 2024-05-10T21:11:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.481+0000 7f577c208b80 1 -- start start 2024-05-10T21:11:26.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.482+0000 7f575f400700 1 -- 172.21.15.50:0/1844568646 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5750003ce0 con 0x7f5768105e50 2024-05-10T21:11:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e4832710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4832c50 0x5611e4833060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5611e4754a20 con 0x5611e4832c50 2024-05-10T21:11:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5611e46f2760 con 0x5611e46b4e40 2024-05-10T21:11:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.484+0000 7f577c208b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5611e47509e0 con 0x5611e4838bf0 2024-05-10T21:11:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.486+0000 7f5767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.486+0000 7f5767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43128/0 (socket says 172.21.15.50:43128) 2024-05-10T21:11:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.486+0000 7f5767400700 1 -- 172.21.15.50:0/2585859628 learned_addr learned my addr 172.21.15.50:0/2585859628 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5766a00700 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e4832710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5766000700 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4832c50 0x5611e4833060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5767400700 1 -- 172.21.15.50:0/2585859628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 msgr2=0x5611e4832710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5767400700 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e4832710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5767400700 1 -- 172.21.15.50:0/2585859628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4832c50 msgr2=0x5611e4833060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.487+0000 7f5767400700 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4832c50 0x5611e4833060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.488+0000 7f5767400700 1 -- 172.21.15.50:0/2585859628 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e46a18d0 con 0x5611e46b4e40 2024-05-10T21:11:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.488+0000 7f5767400700 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5760009400 tx=0x7f5760009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22bb2a26d0ac898d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.489+0000 7f575f400700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 0x7f574c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.489+0000 7f5757e00700 1 -- 172.21.15.50:0/2585859628 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576001a070 con 0x5611e46b4e40 2024-05-10T21:11:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.489+0000 7f575f400700 1 -- 172.21.15.50:0/1844568646 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f57500e32a0 con 0x7f5768105e50 2024-05-10T21:11:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.489+0000 7f5757e00700 1 -- 172.21.15.50:0/2585859628 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5760015aa0 con 0x5611e46b4e40 2024-05-10T21:11:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.489+0000 7f5757e00700 1 -- 172.21.15.50:0/2585859628 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760014950 con 0x5611e46b4e40 2024-05-10T21:11:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.490+0000 7f5766a00700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 0x7f574c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.491+0000 7f577c208b80 1 -- 172.21.15.50:0/2585859628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 msgr2=0x5611e48386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.491+0000 7f577c208b80 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5760009400 tx=0x7f5760009920 comp rx=0 tx=0).stop 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.492+0000 7f577c208b80 1 -- 172.21.15.50:0/2585859628 shutdown_connections 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.492+0000 7f577c208b80 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e48386b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.492+0000 7f577c208b80 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4832c50 0x5611e4833060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.492+0000 7f577c208b80 1 --2- 172.21.15.50:0/2585859628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e4832710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.492+0000 7f577c208b80 1 -- 172.21.15.50:0/2585859628 >> 172.21.15.50:0/2585859628 conn(0x5611e46f3a80 msgr2=0x5611e482d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.493+0000 7f577c208b80 1 -- 172.21.15.50:0/2585859628 shutdown_connections 2024-05-10T21:11:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.493+0000 7f577c208b80 1 -- 172.21.15.50:0/2585859628 wait complete. 2024-05-10T21:11:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.494+0000 7f5766a00700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 0x7f574c0b4f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5758005950 tx=0x7f5758009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.494+0000 7f577c208b80 1 -- start start 2024-05-10T21:11:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.494+0000 7f577c208b80 1 Processor -- start 2024-05-10T21:11:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.496+0000 7f577c208b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e483c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.496+0000 7f577c208b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.496+0000 7f577c208b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4838bf0 0x5611e483ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.497+0000 7f577c208b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5611e483f6c0 con 0x5611e4838bf0 2024-05-10T21:11:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.497+0000 7f577c208b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5611e46bab20 con 0x5611e46b4e40 2024-05-10T21:11:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.497+0000 7f577c208b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5611e483f840 con 0x5611e4832c50 2024-05-10T21:11:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.497+0000 7f577c208b80 10 client.0 _ll_register_callbacks cb 0x5611e46e8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47450/0 (socket says 172.21.15.50:47450) 2024-05-10T21:11:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 -- 172.21.15.50:0/2360627422 learned_addr learned my addr 172.21.15.50:0/2360627422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766a00700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e483c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5767400700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4838bf0 0x5611e483ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 -- 172.21.15.50:0/2360627422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 msgr2=0x5611e483c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e483c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 -- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4838bf0 msgr2=0x5611e483ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4838bf0 0x5611e483ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766000700 1 -- 172.21.15.50:0/2360627422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e471a420 con 0x5611e4832c50 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.498+0000 7f5766a00700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e483c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.499+0000 7f5766000700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f57500ab640 tx=0x7f575001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f71f200e5c6d932 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.499+0000 7f5754c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.499+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57500094f0 con 0x5611e4832c50 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.499+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f57500cd0e0 con 0x5611e4832c50 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.500+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5760015aa0 con 0x5611e4832c50 2024-05-10T21:11:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.501+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57500ab1c0 con 0x5611e4832c50 2024-05-10T21:11:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.501+0000 7f577c208b80 10 client.25930 fetch_fsmap learned FSMap version 26 2024-05-10T21:11:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.501+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611e483f770 con 0x5611e4832c50 2024-05-10T21:11:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.501+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5760015aa0 con 0x5611e4832c50 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f57500ab910 con 0x5611e4832c50 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 10 client.25930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 20 client.25930 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e484c8b0 0x5611e484ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 4 client.25930 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.504+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e47ff070 con 0x5611e484c8b0 2024-05-10T21:11:26.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.508+0000 7f5766a00700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e484c8b0 0x5611e484ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.518+0000 7f5766a00700 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e484c8b0 0x5611e484ed00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b83c2ef10d6daed5 server_cookie=f35adb88569e1ab7 in_seq=0 out_seq=0 2024-05-10T21:11:26.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.518+0000 7f5754c00700 10 client.25930 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:11:26.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.554+0000 7f5754c00700 1 -- 172.21.15.50:0/2360627422 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611e47ff070 con 0x5611e484c8b0 2024-05-10T21:11:26.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.554+0000 7f5754c00700 10 client.25930 handle_command_reply: tid=0 2024-05-10T21:11:26.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.554+0000 7f577c208b80 1 client.25930 shutdown 2024-05-10T21:11:26.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.555+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 msgr2=0x5611e483c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.555+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f57500ab640 tx=0x7f575001d000 comp rx=0 tx=0).stop 2024-05-10T21:11:26.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 shutdown_connections 2024-05-10T21:11:26.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e46b4e40 0x5611e483c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e484c8b0 0x5611e484ed00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e4838bf0 0x5611e483ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 --2- 172.21.15.50:0/2360627422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4832c50 0x5611e483c8f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 >> 172.21.15.50:0/2360627422 conn(0x5611e46f3a80 msgr2=0x5611e4831700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f5754c00700 0 client.25930 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:11:26.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 shutdown_connections 2024-05-10T21:11:26.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.560+0000 7f577c208b80 1 -- 172.21.15.50:0/2360627422 wait complete. 2024-05-10T21:11:26.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.561+0000 7f577c208b80 20 client.25930 trim_cache size 0 max 16384 2024-05-10T21:11:26.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 Processor -- start 2024-05-10T21:11:26.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 -- start start 2024-05-10T21:11:26.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4832710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e483cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.604+0000 7f577c208b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57681c0020 con 0x5611e46b4e40 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f577c208b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57681a12e0 con 0x5611e4832c50 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f577c208b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f576819f1e0 con 0x5611e4838bf0 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f5766000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f5766000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43160/0 (socket says 172.21.15.50:43160) 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f5766000700 1 -- 172.21.15.50:0/3110975458 learned_addr learned my addr 172.21.15.50:0/3110975458 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.605+0000 7f5766a00700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4832710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5767400700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e483cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5766a00700 1 -- 172.21.15.50:0/3110975458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 msgr2=0x5611e483cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5766a00700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e483cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5766a00700 1 -- 172.21.15.50:0/3110975458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 msgr2=0x5611e482d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5766a00700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.606+0000 7f5766a00700 1 -- 172.21.15.50:0/3110975458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e4838a20 con 0x5611e46b4e40 2024-05-10T21:11:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.607+0000 7f5766000700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.607+0000 7f5766a00700 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4832710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5611e484ccc0 tx=0x7f5758017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35984dafedc83f26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.607+0000 7f5757400700 1 -- 172.21.15.50:0/3110975458 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758020030 con 0x5611e46b4e40 2024-05-10T21:11:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.608+0000 7f5757400700 1 -- 172.21.15.50:0/3110975458 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575802ce50 con 0x5611e46b4e40 2024-05-10T21:11:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.608+0000 7f5757400700 1 -- 172.21.15.50:0/3110975458 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575802c9b0 con 0x5611e46b4e40 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.609+0000 7f577c208b80 1 -- 172.21.15.50:0/3110975458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 msgr2=0x5611e4832710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.609+0000 7f577c208b80 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4832710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5611e484ccc0 tx=0x7f5758017000 comp rx=0 tx=0).stop 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 -- 172.21.15.50:0/3110975458 shutdown_connections 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e483cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4832710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 --2- 172.21.15.50:0/3110975458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e482d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 -- 172.21.15.50:0/3110975458 >> 172.21.15.50:0/3110975458 conn(0x5611e46f3a80 msgr2=0x5611e4837b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.610+0000 7f577c208b80 1 -- 172.21.15.50:0/3110975458 shutdown_connections 2024-05-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.611+0000 7f577c208b80 1 -- 172.21.15.50:0/3110975458 wait complete. 2024-05-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.611+0000 7f577c208b80 1 -- start start 2024-05-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.612+0000 7f577c208b80 1 Processor -- start 2024-05-10T21:11:26.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4853f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e4854490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f576819f7a0 con 0x5611e46b4e40 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57681a0c10 con 0x5611e4832c50 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f576819fe70 con 0x5611e4838bf0 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.613+0000 7f577c208b80 10 client.0 _ll_register_callbacks cb 0x5611e46e8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47468/0 (socket says 172.21.15.50:47468) 2024-05-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 -- 172.21.15.50:0/2706924121 learned_addr learned my addr 172.21.15.50:0/2706924121 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5766a00700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4853f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 -- 172.21.15.50:0/2706924121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 msgr2=0x5611e4854490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e4854490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 -- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 msgr2=0x5611e4853f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4853f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5767400700 1 -- 172.21.15.50:0/2706924121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e4838240 con 0x5611e4838bf0 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.614+0000 7f5766a00700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4853f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.616+0000 7f5767400700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5611e4833a30 tx=0x7f5760015bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93cf85496b3386e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.616+0000 7f5754c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.616+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576001a070 con 0x5611e4838bf0 2024-05-10T21:11:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.616+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7f5760004c70 con 0x5611e4838bf0 2024-05-10T21:11:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.616+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575802ce50 con 0x5611e4838bf0 2024-05-10T21:11:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.617+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5760006040 con 0x5611e4838bf0 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.617+0000 7f577c208b80 10 client.25936 fetch_fsmap learned FSMap version 26 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.617+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611e48b7270 con 0x5611e4838bf0 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.617+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f575802ce50 con 0x5611e4838bf0 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.617+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f5760027030 con 0x5611e4838bf0 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 10 client.25936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 20 client.25936 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e48b76a0 0x5611e48b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 4 client.25936 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611e4762400 con 0x5611e48b76a0 2024-05-10T21:11:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.618+0000 7f5766a00700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e48b76a0 0x5611e48b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.619+0000 7f5766a00700 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e48b76a0 0x5611e48b9ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=564fb5493cd062e1 server_cookie=6eaca6b25738cad in_seq=0 out_seq=0 2024-05-10T21:11:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.619+0000 7f5754c00700 10 client.25936 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:11:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.632+0000 7f5754c00700 1 -- 172.21.15.50:0/2706924121 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611e4762400 con 0x5611e48b76a0 2024-05-10T21:11:26.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.632+0000 7f5754c00700 10 client.25936 handle_command_reply: tid=0 2024-05-10T21:11:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.633+0000 7f577c208b80 1 client.25936 shutdown 2024-05-10T21:11:26.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.634+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 msgr2=0x5611e48549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.634+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5611e4833a30 tx=0x7f5760015bb0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 shutdown_connections 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5611e48b76a0 0x5611e48b9ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611e4838bf0 0x5611e48549d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611e46b4e40 0x5611e4853f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 --2- 172.21.15.50:0/2706924121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611e4832c50 0x5611e4854490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 >> 172.21.15.50:0/2706924121 conn(0x5611e46f3a80 msgr2=0x5611e4837b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f5754c00700 0 client.25936 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:11:26.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.635+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 shutdown_connections 2024-05-10T21:11:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.636+0000 7f577c208b80 1 -- 172.21.15.50:0/2706924121 wait complete. 2024-05-10T21:11:26.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.636+0000 7f577c208b80 20 client.25936 trim_cache size 0 max 16384 2024-05-10T21:11:26.635 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:11:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.639+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 msgr2=0x7f574c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.639+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 0x7f574c0b4f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5758005950 tx=0x7f5758009310 comp rx=0 tx=0).stop 2024-05-10T21:11:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.639+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 msgr2=0x7f57681a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.639+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 0x7f57681a26b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f575000c010 tx=0x7f575000fab0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 shutdown_connections 2024-05-10T21:11:26.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f574c0b2a50 0x7f574c0b4f60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5768105500 0x7f57681a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576810e8e0 0x7f57681ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 --2- 172.21.15.50:0/1844568646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768105e50 0x7f57681a26b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:26.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.640+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 >> 172.21.15.50:0/1844568646 conn(0x7f57680fedc0 msgr2=0x7f57680ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:26.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.643+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 shutdown_connections 2024-05-10T21:11:26.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:26.644+0000 7f576d600700 1 -- 172.21.15.50:0/1844568646 wait complete. 2024-05-10T21:11:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 -- 172.21.15.50:0/2188700238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50104560 msgr2=0x7fef50106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 --2- 172.21.15.50:0/2188700238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50104560 0x7fef50106980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef44009900 tx=0x7fef4402a410 comp rx=0 tx=0).stop 2024-05-10T21:11:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 -- 172.21.15.50:0/2188700238 shutdown_connections 2024-05-10T21:11:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 --2- 172.21.15.50:0/2188700238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef501092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 --2- 172.21.15.50:0/2188700238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50109820 0x7fef5010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 --2- 172.21.15.50:0/2188700238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50104560 0x7fef50106980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.102+0000 7fef56e00640 1 -- 172.21.15.50:0/2188700238 >> 172.21.15.50:0/2188700238 conn(0x7fef500fdb60 msgr2=0x7fef50100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.103+0000 7fef56e00640 1 -- 172.21.15.50:0/2188700238 shutdown_connections 2024-05-10T21:11:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.103+0000 7fef56e00640 1 -- 172.21.15.50:0/2188700238 wait complete. 2024-05-10T21:11:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 Processor -- start 2024-05-10T21:11:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 -- start start 2024-05-10T21:11:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 0x7fef5019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef5019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 0x7fef501a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010f570 con 0x7fef50109820 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010f3f0 con 0x7fef50104560 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.104+0000 7fef56e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010f6f0 con 0x7fef50106ec0 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 0x7fef5019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef5019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 0x7fef5019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43188/0 (socket says 172.21.15.50:43188) 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55a00640 1 -- 172.21.15.50:0/2648059204 learned_addr learned my addr 172.21.15.50:0/2648059204 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef56400640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 0x7fef501a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 -- 172.21.15.50:0/2648059204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 msgr2=0x7fef5019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 0x7fef5019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 -- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 msgr2=0x7fef501a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 0x7fef501a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef55000640 1 -- 172.21.15.50:0/2648059204 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef501a3f40 con 0x7fef50106ec0 2024-05-10T21:11:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.105+0000 7fef56400640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 0x7fef501a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef55000640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef5019c940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9f0 tx=0x7fef3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef4e000640 1 -- 172.21.15.50:0/2648059204 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00ccc0 con 0x7fef50106ec0 2024-05-10T21:11:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef4e000640 1 -- 172.21.15.50:0/2648059204 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3c005020 con 0x7fef50106ec0 2024-05-10T21:11:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef501a4230 con 0x7fef50106ec0 2024-05-10T21:11:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef4e000640 1 -- 172.21.15.50:0/2648059204 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c004a00 con 0x7fef50106ec0 2024-05-10T21:11:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.106+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a4780 con 0x7fef50106ec0 2024-05-10T21:11:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.110+0000 7fef5f46b740 1 Processor -- start 2024-05-10T21:11:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 -- start start 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 0x559b65c1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 0x559b65c23780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b65b80550 con 0x559b658594d0 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b65b25bc0 con 0x559b6587a1b0 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef5f46b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b65b7c510 con 0x559b658512f0 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef56400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef56400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43202/0 (socket says 172.21.15.50:43202) 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef56400640 1 -- 172.21.15.50:0/2814749415 learned_addr learned my addr 172.21.15.50:0/2814749415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.111+0000 7fef4e000640 1 -- 172.21.15.50:0/2648059204 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef3c004ba0 con 0x7fef50106ec0 2024-05-10T21:11:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef56400640 1 -- 172.21.15.50:0/2814749415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 msgr2=0x559b65c1c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef55a00640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 0x559b65c1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef55000640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 0x559b65c23780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef56400640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 0x559b65c1c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef56400640 1 -- 172.21.15.50:0/2814749415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 msgr2=0x559b65c23780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef56400640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 0x559b65c23780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef56400640 1 -- 172.21.15.50:0/2814749415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b65892210 con 0x559b6587a1b0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef55a00640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 0x559b65c1c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.112+0000 7fef55000640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 0x559b65c23780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.113+0000 7fef56400640 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef4800f360 tx=0x7fef4800f820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c779c2faf807733 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.113+0000 7fef42a00640 1 -- 172.21.15.50:0/2814749415 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48015070 con 0x559b6587a1b0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.113+0000 7fef42a00640 1 -- 172.21.15.50:0/2814749415 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef48004d20 con 0x559b6587a1b0 2024-05-10T21:11:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.113+0000 7fef42a00640 1 -- 172.21.15.50:0/2814749415 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef480053e0 con 0x559b6587a1b0 2024-05-10T21:11:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.114+0000 7fef4e000640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 0x7fef380b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.114+0000 7fef4e000640 1 -- 172.21.15.50:0/2648059204 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fef3c0e21f0 con 0x7fef50106ec0 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.114+0000 7fef55a00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 0x7fef380b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.114+0000 7fef5f46b740 1 -- 172.21.15.50:0/2814749415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 msgr2=0x559b65c1c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.114+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef4800f360 tx=0x7fef4800f820 comp rx=0 tx=0).stop 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 -- 172.21.15.50:0/2814749415 shutdown_connections 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658512f0 0x559b65c1c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658594d0 0x559b65c23780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2814749415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b6587a1b0 0x559b65c1c300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 -- 172.21.15.50:0/2814749415 >> 172.21.15.50:0/2814749415 conn(0x559b6586e300 msgr2=0x559b65c17c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 -- 172.21.15.50:0/2814749415 shutdown_connections 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef5f46b740 1 -- 172.21.15.50:0/2814749415 wait complete. 2024-05-10T21:11:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.115+0000 7fef55a00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 0x7fef380b4d70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x559b65c1d6e0 tx=0x7fef44005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.116+0000 7fef5f46b740 1 -- start start 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.116+0000 7fef5f46b740 1 Processor -- start 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b65c241d0 con 0x559b658512f0 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559b65b39660 con 0x559b658594d0 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b65c24350 con 0x559b6587a1b0 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.117+0000 7fef5f46b740 10 client.0 _ll_register_callbacks cb 0x559b65b03400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef55a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef55a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c25ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54588/0 (socket says 172.21.15.50:54588) 2024-05-10T21:11:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef55a00640 1 -- 172.21.15.50:0/2904558233 learned_addr learned my addr 172.21.15.50:0/2904558233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef55000640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 -- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 msgr2=0x559b65c26420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c26420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 -- 172.21.15.50:0/2904558233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 msgr2=0x559b65c25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c25ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.118+0000 7fef56400640 1 -- 172.21.15.50:0/2904558233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6595e620 con 0x559b6587a1b0 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.119+0000 7fef56400640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef48006010 tx=0x7fef48004c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0c4142acf13746 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.119+0000 7fef37400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.119+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48015040 con 0x559b6587a1b0 2024-05-10T21:11:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.119+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fef48011910 con 0x559b6587a1b0 2024-05-10T21:11:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.119+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef48004d20 con 0x559b6587a1b0 2024-05-10T21:11:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef48011b10 con 0x559b6587a1b0 2024-05-10T21:11:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 10 client.25945 fetch_fsmap learned FSMap version 26 2024-05-10T21:11:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b65c30190 con 0x559b6587a1b0 2024-05-10T21:11:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef48004d20 con 0x559b6587a1b0 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7fef4802c030 con 0x559b6587a1b0 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 10 client.25945 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 20 client.25945 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b65838d80 0x559b65c335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.120+0000 7fef5f46b740 4 client.25945 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.121+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b65b8e160 con 0x559b65838d80 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.121+0000 7fef55a00640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b65838d80 0x559b65c335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.121+0000 7fef37400640 10 client.25945 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.121+0000 7fef55a00640 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b65838d80 0x559b65c335b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54dc8911620e3aa server_cookie=bd796723e4e20bf3 in_seq=0 out_seq=0 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.122+0000 7fef37400640 1 -- 172.21.15.50:0/2904558233 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b65b8e160 con 0x559b65838d80 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.122+0000 7fef37400640 10 client.25945 handle_command_reply: tid=0 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.122+0000 7fef5f46b740 1 client.25945 shutdown 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 msgr2=0x559b65c2d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef48006010 tx=0x7fef48004c30 comp rx=0 tx=0).stop 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 shutdown_connections 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b65838d80 0x559b65c335b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d380 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c26420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2904558233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 >> 172.21.15.50:0/2904558233 conn(0x559b6586e300 msgr2=0x559b65c16c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef37400640 0 client.25945 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.123+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 shutdown_connections 2024-05-10T21:11:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.124+0000 7fef5f46b740 1 -- 172.21.15.50:0/2904558233 wait complete. 2024-05-10T21:11:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.124+0000 7fef5f46b740 20 client.25945 trim_cache size 0 max 16384 2024-05-10T21:11:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 Processor -- start 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 -- start start 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef501afc10 con 0x559b658512f0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ad4e0 con 0x559b658594d0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.144+0000 7fef5f46b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ab330 con 0x559b6587a1b0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef56400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef56400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47528/0 (socket says 172.21.15.50:47528) 2024-05-10T21:11:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef56400640 1 -- 172.21.15.50:0/2080891786 learned_addr learned my addr 172.21.15.50:0/2080891786 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43224/0 (socket says 172.21.15.50:43224) 2024-05-10T21:11:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 -- 172.21.15.50:0/2080891786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 msgr2=0x559b65c2d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 -- 172.21.15.50:0/2080891786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 msgr2=0x559b65c24e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c24e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 -- 172.21.15.50:0/2080891786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b65c23710 con 0x559b658594d0 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef56400640 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.145+0000 7fef55000640 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0a94e0 tx=0x7fef3c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8146670d6517c0e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.146+0000 7fef42000640 1 -- 172.21.15.50:0/2080891786 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c01c070 con 0x559b658594d0 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.146+0000 7fef42000640 1 -- 172.21.15.50:0/2080891786 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3c0e1030 con 0x559b658594d0 2024-05-10T21:11:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.146+0000 7fef42000640 1 -- 172.21.15.50:0/2080891786 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c0a5a70 con 0x559b658594d0 2024-05-10T21:11:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 -- 172.21.15.50:0/2080891786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 msgr2=0x559b65c2d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0a94e0 tx=0x7fef3c009f60 comp rx=0 tx=0).stop 2024-05-10T21:11:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 -- 172.21.15.50:0/2080891786 shutdown_connections 2024-05-10T21:11:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6587a1b0 0x559b65c2d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658594d0 0x559b65c2d2a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 --2- 172.21.15.50:0/2080891786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b658512f0 0x559b65c24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 -- 172.21.15.50:0/2080891786 >> 172.21.15.50:0/2080891786 conn(0x559b6586e300 msgr2=0x559b65c246f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 -- 172.21.15.50:0/2080891786 shutdown_connections 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.147+0000 7fef5f46b740 1 -- 172.21.15.50:0/2080891786 wait complete. 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.148+0000 7fef5f46b740 1 -- start start 2024-05-10T21:11:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.148+0000 7fef5f46b740 1 Processor -- start 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.148+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658594d0 0x559b65c2b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 0x559b65c998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ab9a0 con 0x559b6587a1b0 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ace10 con 0x559b658512f0 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef501ac070 con 0x559b658594d0 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef5f46b740 10 client.0 _ll_register_callbacks cb 0x7fef48009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef56400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 0x559b65c998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43240/0 (socket says 172.21.15.50:43240) 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 -- 172.21.15.50:0/1937712064 learned_addr learned my addr 172.21.15.50:0/1937712064 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55000640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658594d0 0x559b65c2b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 -- 172.21.15.50:0/1937712064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658594d0 msgr2=0x559b65c2b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658594d0 0x559b65c2b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 -- 172.21.15.50:0/1937712064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 msgr2=0x559b65c998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 0x559b65c998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.149+0000 7fef55a00640 1 -- 172.21.15.50:0/1937712064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b65c99e80 con 0x559b658512f0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef56400640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 0x559b65c998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef55a00640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef44005940 tx=0x7fef44009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee82b705d5ae737 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef37400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44054510 con 0x559b658512f0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef3c0e1030 con 0x559b658512f0 2024-05-10T21:11:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.150+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8177+0+0 (secure 0 0 0) 0x7fef440546b0 con 0x559b658512f0 2024-05-10T21:11:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.151+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef440548b0 con 0x559b658512f0 2024-05-10T21:11:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.151+0000 7fef5f46b740 10 client.25985 fetch_fsmap learned FSMap version 26 2024-05-10T21:11:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.151+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b65c9b1a0 con 0x559b658512f0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.151+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3c0e1030 con 0x559b658512f0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.151+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7fef4405b070 con 0x559b658512f0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 10 client.25985 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 20 client.25985 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b6580e1c0 0x559b65c9dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 4 client.25985 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b65b87fd0 con 0x559b6580e1c0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.152+0000 7fef55000640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b6580e1c0 0x559b65c9dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.153+0000 7fef55000640 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b6580e1c0 0x559b65c9dae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8775e4adfc1b7aca server_cookie=83fe11372a4271d5 in_seq=0 out_seq=0 2024-05-10T21:11:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.153+0000 7fef37400640 10 client.25985 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:27.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:26 smithi050 ceph-mon[29546]: pgmap v537: 97 pgs: 97 active+clean; 5.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 367 KiB/s wr, 106 op/s 2024-05-10T21:11:27.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:26 smithi092 ceph-mon[34415]: pgmap v537: 97 pgs: 97 active+clean; 5.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 367 KiB/s wr, 106 op/s 2024-05-10T21:11:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:26 smithi142 ceph-mon[34846]: pgmap v537: 97 pgs: 97 active+clean; 5.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 367 KiB/s wr, 106 op/s 2024-05-10T21:11:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef37400640 1 -- 172.21.15.50:0/1937712064 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b65b87fd0 con 0x559b6580e1c0 2024-05-10T21:11:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef37400640 10 client.25985 handle_command_reply: tid=0 2024-05-10T21:11:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef5f46b740 1 client.25985 shutdown 2024-05-10T21:11:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 msgr2=0x559b65c2abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef44005940 tx=0x7fef44009260 comp rx=0 tx=0).stop 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.426+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 shutdown_connections 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559b6580e1c0 0x559b65c9dae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b658594d0 0x559b65c2b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b6587a1b0 0x559b65c998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 --2- 172.21.15.50:0/1937712064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559b658512f0 0x559b65c2abd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef37400640 0 client.25985 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 >> 172.21.15.50:0/1937712064 conn(0x559b6586e300 msgr2=0x559b65c1aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 shutdown_connections 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 1 -- 172.21.15.50:0/1937712064 wait complete. 2024-05-10T21:11:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.427+0000 7fef5f46b740 20 client.25985 trim_cache size 0 max 16384 2024-05-10T21:11:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:11:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:11:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:11:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 msgr2=0x7fef380b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 0x7fef380b4d70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x559b65c1d6e0 tx=0x7fef44005e20 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 msgr2=0x7fef5019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef5019c940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e9f0 tx=0x7fef3c00eeb0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 shutdown_connections 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef380b2860 0x7fef380b4d70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef50106ec0 0x7fef5019c940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef50109820 0x7fef501a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 --2- 172.21.15.50:0/2648059204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef50104560 0x7fef5019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.428+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 >> 172.21.15.50:0/2648059204 conn(0x7fef500fdb60 msgr2=0x7fef50101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.429+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 shutdown_connections 2024-05-10T21:11:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:27.429+0000 7fef56e00640 1 -- 172.21.15.50:0/2648059204 wait complete. 2024-05-10T21:11:27.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:11:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:28.367 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:11:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setuser ceph since I am not root 2024-05-10T21:11:28.367 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:11:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setgroup ceph since I am not root 2024-05-10T21:11:28.367 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:11:28 smithi092 ceph-mds[64505]: -- 172.21.15.92:0/2343703600 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a027c9a40 con 0x561a02811600 2024-05-10T21:11:28.367 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:11:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: starting mds.h at 2024-05-10T21:11:29.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:28 smithi050 ceph-mon[29546]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-10T21:11:29.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:28 smithi050 ceph-mon[29546]: daemon mds.h restarted 2024-05-10T21:11:29.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:28 smithi092 ceph-mon[34415]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-10T21:11:29.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:28 smithi092 ceph-mon[34415]: daemon mds.h restarted 2024-05-10T21:11:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:28 smithi142 ceph-mon[34846]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-05-10T21:11:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:28 smithi142 ceph-mon[34846]: daemon mds.h restarted 2024-05-10T21:11:30.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:29 smithi050 ceph-mon[29546]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T21:11:30.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:29 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6836/1113302816,v1:172.21.15.92:6837/1113302816] up:boot 2024-05-10T21:11:30.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:29 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:11:30.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:29 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:11:30.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:29 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:29 smithi092 ceph-mon[34415]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T21:11:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:29 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6836/1113302816,v1:172.21.15.92:6837/1113302816] up:boot 2024-05-10T21:11:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:29 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:11:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:29 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:11:30.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:29 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:29 smithi142 ceph-mon[34846]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T21:11:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:29 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6836/1113302816,v1:172.21.15.92:6837/1113302816] up:boot 2024-05-10T21:11:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:29 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:11:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:29 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:11:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:29 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.477+0000 7f2d0e000700 1 -- 172.21.15.50:0/580844333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08110970 msgr2=0x7f2d08110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.477+0000 7f2d0e000700 1 --2- 172.21.15.50:0/580844333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08110970 0x7f2d08110d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009f90 tx=0x7f2cf800b330 comp rx=0 tx=0).stop 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.477+0000 7f2d0e000700 1 -- 172.21.15.50:0/580844333 shutdown_connections 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.478+0000 7f2d0e000700 1 --2- 172.21.15.50:0/580844333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08110970 0x7f2d08110d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.478+0000 7f2d0e000700 1 --2- 172.21.15.50:0/580844333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810b2e0 0x7f2d08117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.478+0000 7f2d0e000700 1 --2- 172.21.15.50:0/580844333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d0810a990 0x7f2d0810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.478+0000 7f2d0e000700 1 -- 172.21.15.50:0/580844333 >> 172.21.15.50:0/580844333 conn(0x7f2d08077c80 msgr2=0x7f2d080780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.478+0000 7f2d0e000700 1 -- 172.21.15.50:0/580844333 shutdown_connections 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.480+0000 7f2d0e000700 1 -- 172.21.15.50:0/580844333 wait complete. 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.482+0000 7f2d0e000700 1 Processor -- start 2024-05-10T21:11:30.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.483+0000 7f2d0e000700 1 -- start start 2024-05-10T21:11:30.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.484+0000 7f2d0e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810a990 0x7f2d081176b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.484+0000 7f2d0e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 0x7f2d081137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.484+0000 7f2d0e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.485+0000 7f2d0e000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0807f8c0 con 0x7f2d08110970 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.485+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.485+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54628/0 (socket says 172.21.15.50:54628) 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.485+0000 7f2d07e00700 1 -- 172.21.15.50:0/1274294478 learned_addr learned my addr 172.21.15.50:0/1274294478 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.485+0000 7f2d07400700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810a990 0x7f2d081176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.486+0000 7f2cffe00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 0x7f2d081137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.488+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0807f740 con 0x7f2d0810b2e0 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.488+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0807fa40 con 0x7f2d0810a990 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2d07e00700 1 -- 172.21.15.50:0/1274294478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810a990 msgr2=0x7f2d081176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2d07e00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810a990 0x7f2d081176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2d07e00700 1 -- 172.21.15.50:0/1274294478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 msgr2=0x7f2d081137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2d07e00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 0x7f2d081137f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2d07e00700 1 -- 172.21.15.50:0/1274294478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d08113d30 con 0x7f2d08110970 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.490+0000 7f2cffe00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 0x7f2d081137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.491+0000 7f2d07e00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b4a0 tx=0x7f2d0000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.492+0000 7f2d04c00700 1 -- 172.21.15.50:0/1274294478 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00015020 con 0x7f2d08110970 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.492+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d08114020 con 0x7f2d08110970 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.493+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d08114560 con 0x7f2d08110970 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.493+0000 7f2d04c00700 1 -- 172.21.15.50:0/1274294478 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00011ce0 con 0x7f2d08110970 2024-05-10T21:11:30.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.493+0000 7f2d04c00700 1 -- 172.21.15.50:0/1274294478 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00015020 con 0x7f2d08110970 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.512+0000 7f2d1caabb80 1 Processor -- start 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.513+0000 7f2d04c00700 1 -- 172.21.15.50:0/1274294478 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d00018020 con 0x7f2d08110970 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.513+0000 7f2d1caabb80 1 -- start start 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.513+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33bcb2e40 0x55e33be366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.513+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.514+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be30c50 0x55e33be31060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.514+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e33bd52a20 con 0x55e33be36bf0 2024-05-10T21:11:30.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.514+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e33bcf0760 con 0x55e33be30c50 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.514+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e33bd4e9e0 con 0x55e33bcb2e40 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.515+0000 7f2d07400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.515+0000 7f2d07400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54630/0 (socket says 172.21.15.50:54630) 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.515+0000 7f2d07400700 1 -- 172.21.15.50:0/771007503 learned_addr learned my addr 172.21.15.50:0/771007503 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 -- 172.21.15.50:0/771007503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33bcb2e40 msgr2=0x55e33be366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33bcb2e40 0x55e33be366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 -- 172.21.15.50:0/771007503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be30c50 msgr2=0x55e33be31060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be30c50 0x55e33be31060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 -- 172.21.15.50:0/771007503 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33bc9f8d0 con 0x55e33be36bf0 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.516+0000 7f2d07400700 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009470 tx=0x7f2cf80094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede62ea50e6bb0db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.521+0000 7f2cfcc00700 1 -- 172.21.15.50:0/771007503 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800be20 con 0x55e33be36bf0 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.521+0000 7f2cfcc00700 1 -- 172.21.15.50:0/771007503 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf8007500 con 0x55e33be36bf0 2024-05-10T21:11:30.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.521+0000 7f2cfcc00700 1 -- 172.21.15.50:0/771007503 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf801c6a0 con 0x55e33be36bf0 2024-05-10T21:11:30.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.523+0000 7f2d1caabb80 1 -- 172.21.15.50:0/771007503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 msgr2=0x55e33be30710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.523+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009470 tx=0x7f2cf80094a0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.524+0000 7f2d04c00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 0x7f2cec0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.524+0000 7f2d04c00700 1 -- 172.21.15.50:0/1274294478 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2d000e1450 con 0x7f2d08110970 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.525+0000 7f2cffe00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 0x7f2cec0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.526+0000 7f2d1caabb80 1 -- 172.21.15.50:0/771007503 shutdown_connections 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.526+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be30c50 0x55e33be31060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.526+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33bcb2e40 0x55e33be366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.526+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/771007503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be30710 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.526+0000 7f2d1caabb80 1 -- 172.21.15.50:0/771007503 >> 172.21.15.50:0/771007503 conn(0x55e33bcf1a80 msgr2=0x55e33be2b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.528+0000 7f2d1caabb80 1 -- 172.21.15.50:0/771007503 shutdown_connections 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.528+0000 7f2cffe00700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 0x7f2cec0b5060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0005fd0 tx=0x7f2cf001b440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.528+0000 7f2d1caabb80 1 -- 172.21.15.50:0/771007503 wait complete. 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.529+0000 7f2d1caabb80 1 -- start start 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.529+0000 7f2d1caabb80 1 Processor -- start 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.531+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.531+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.531+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.531+0000 7f2d07400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.531+0000 7f2d07400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43274/0 (socket says 172.21.15.50:43274) 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2cffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2cffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47586/0 (socket says 172.21.15.50:47586) 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54642/0 (socket says 172.21.15.50:54642) 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e33be3d660 con 0x55e33be36bf0 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e33bcb8b20 con 0x55e33bcb2e40 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.532+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e33be3d7e0 con 0x55e33be30c50 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d1caabb80 10 client.0 _ll_register_callbacks cb 0x55e33bce6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 -- 172.21.15.50:0/398296195 learned_addr learned my addr 172.21.15.50:0/398296195 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 -- 172.21.15.50:0/398296195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 msgr2=0x55e33be3a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 -- 172.21.15.50:0/398296195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 msgr2=0x55e33be3ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.533+0000 7f2d07400700 1 -- 172.21.15.50:0/398296195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33bd18420 con 0x55e33bcb2e40 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.534+0000 7f2d07e00700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.534+0000 7f2cf5600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.534+0000 7f2d07400700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009960 tx=0x7f2cf80126d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a7bdd0f0a02bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.534+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8028070 con 0x55e33bcb2e40 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.534+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2cf800f040 con 0x55e33bcb2e40 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.535+0000 7f2cffe00700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T21:11:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.535+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf8007500 con 0x55e33bcb2e40 2024-05-10T21:11:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.537+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cf8028070 con 0x55e33bcb2e40 2024-05-10T21:11:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.537+0000 7f2d1caabb80 10 client.26000 fetch_fsmap learned FSMap version 29 2024-05-10T21:11:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.537+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e33be48230 con 0x55e33bcb2e40 2024-05-10T21:11:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.537+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf8012da0 con 0x55e33bcb2e40 2024-05-10T21:11:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2cf8011070 con 0x55e33bcb2e40 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 20 client.26000 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33be4a700 0x55e33be4cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 4 client.26000 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.539+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2cf8007500 con 0x55e33be4a700 2024-05-10T21:11:30.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.540+0000 7f2cffe00700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33be4a700 0x55e33be4cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.540+0000 7f2cffe00700 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33be4a700 0x55e33be4cb20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696c0372befe4afa server_cookie=fa7d5cf58ed2528a in_seq=0 out_seq=0 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.541+0000 7f2cf5600700 10 client.26000 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.542+0000 7f2cf5600700 1 -- 172.21.15.50:0/398296195 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2cf8007500 con 0x55e33be4a700 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.542+0000 7f2cf5600700 10 client.26000 handle_command_reply: tid=0 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.542+0000 7f2d1caabb80 1 client.26000 shutdown 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 msgr2=0x55e33be3a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009960 tx=0x7f2cf80126d0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 shutdown_connections 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33be4a700 0x55e33be4cb20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33bcb2e40 0x55e33be3a320 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009960 tx=0x7f2cf80126d0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be36bf0 0x55e33be3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/398296195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be30c50 0x55e33be3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.546+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 >> 172.21.15.50:0/398296195 conn(0x55e33bcf1a80 msgr2=0x55e33be2f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.547+0000 7f2cf5600700 0 client.26000 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:30.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.547+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 shutdown_connections 2024-05-10T21:11:30.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.547+0000 7f2d1caabb80 1 -- 172.21.15.50:0/398296195 wait complete. 2024-05-10T21:11:30.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.548+0000 7f2d1caabb80 20 client.26000 trim_cache size 0 max 16384 2024-05-10T21:11:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.598+0000 7f2d1caabb80 1 Processor -- start 2024-05-10T21:11:30.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.598+0000 7f2d1caabb80 1 -- start start 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be36bf0 0x55e33be46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400fa60 con 0x55e33be30c50 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400d290 con 0x55e33be34a30 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400b190 con 0x55e33be36bf0 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43288/0 (socket says 172.21.15.50:43288) 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.599+0000 7f2d07e00700 1 -- 172.21.15.50:0/420858608 learned_addr learned my addr 172.21.15.50:0/420858608 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:30.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.600+0000 7f2cffe00700 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be36bf0 0x55e33be46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.601+0000 7f2d07400700 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.602+0000 7f2d07e00700 1 -- 172.21.15.50:0/420858608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be36bf0 msgr2=0x55e33be46ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.602+0000 7f2d07e00700 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be36bf0 0x55e33be46ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.602+0000 7f2d07e00700 1 -- 172.21.15.50:0/420858608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 msgr2=0x55e33be30710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.602+0000 7f2d07e00700 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be30710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.602+0000 7f2d07e00700 1 -- 172.21.15.50:0/420858608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33be3d5a0 con 0x55e33be34a30 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.603+0000 7f2d07e00700 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d000a9820 tx=0x7f2d000a5b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12b8d76dffe40db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.603+0000 7f2cf7e00700 1 -- 172.21.15.50:0/420858608 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000a9b20 con 0x55e33be34a30 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.603+0000 7f2cf7e00700 1 -- 172.21.15.50:0/420858608 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d000aae50 con 0x55e33be34a30 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.603+0000 7f2cf7e00700 1 -- 172.21.15.50:0/420858608 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000a9d80 con 0x55e33be34a30 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.604+0000 7f2d1caabb80 1 -- 172.21.15.50:0/420858608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 msgr2=0x55e33be3ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.604+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d000a9820 tx=0x7f2d000a5b10 comp rx=0 tx=0).stop 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 -- 172.21.15.50:0/420858608 shutdown_connections 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be34a30 0x55e33be3ac10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be30710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/420858608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be36bf0 0x55e33be46ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 -- 172.21.15.50:0/420858608 >> 172.21.15.50:0/420858608 conn(0x55e33bcf1a80 msgr2=0x55e33be35430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 -- 172.21.15.50:0/420858608 shutdown_connections 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.609+0000 7f2d1caabb80 1 -- 172.21.15.50:0/420858608 wait complete. 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.610+0000 7f2d1caabb80 1 -- start start 2024-05-10T21:11:30.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.610+0000 7f2d1caabb80 1 Processor -- start 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be4e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be34a30 0x55e33be4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400b750 con 0x55e33be30c50 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400cbc0 con 0x55e33be36bf0 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.611+0000 7f2d1caabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400be20 con 0x55e33be34a30 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.612+0000 7f2d1caabb80 10 client.0 _ll_register_callbacks cb 0x55e33bce6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.612+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.612+0000 7f2d07e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43304/0 (socket says 172.21.15.50:43304) 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.612+0000 7f2d07e00700 1 -- 172.21.15.50:0/3404153603 learned_addr learned my addr 172.21.15.50:0/3404153603 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.614+0000 7f2d07e00700 1 -- 172.21.15.50:0/3404153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be34a30 msgr2=0x55e33be4e640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.614+0000 7f2d07e00700 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be34a30 0x55e33be4e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.614+0000 7f2d07e00700 1 -- 172.21.15.50:0/3404153603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 msgr2=0x55e33be4e100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:11:30.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.614+0000 7f2d07e00700 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be4e100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.614+0000 7f2d07e00700 1 -- 172.21.15.50:0/3404153603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33be4ee60 con 0x55e33be36bf0 2024-05-10T21:11:30.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.615+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d000aae50 con 0x55e33be36bf0 2024-05-10T21:11:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.618+0000 7f2d07e00700 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d000a6d90 tx=0x7f2d000aa330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f7e9936e320e59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.618+0000 7f2cf5600700 10 client.26006 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:30.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.618+0000 7f2cf5600700 1 -- 172.21.15.50:0/3404153603 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000c8070 con 0x55e33be36bf0 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.618+0000 7f2cf5600700 1 -- 172.21.15.50:0/3404153603 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2d000a9d80 con 0x55e33be36bf0 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.619+0000 7f2cf5600700 1 -- 172.21.15.50:0/3404153603 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d000a9fe0 con 0x55e33be36bf0 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.620+0000 7f2d1caabb80 10 client.26006 fetch_fsmap learned FSMap version 29 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.620+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55e33bed7020 con 0x55e33be36bf0 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.620+0000 7f2cf5600700 1 -- 172.21.15.50:0/3404153603 <== mon.2 v2:172.21.15.92:3300/0 4 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2d000b24f0 con 0x55e33be36bf0 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.621+0000 7f2d1caabb80 10 client.26006 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.621+0000 7f2d1caabb80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.621+0000 7f2d1caabb80 20 client.26006 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:30.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.621+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33bed7530 0x55e33bed99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.624+0000 7f2d07400700 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33bed7530 0x55e33bed99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.624+0000 7f2d1caabb80 4 client.26006 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:11:30.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.624+0000 7f2d07400700 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33bed7530 0x55e33bed99b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb850bc4c97f0a14 server_cookie=236eebfb80d290d8 in_seq=0 out_seq=0 2024-05-10T21:11:30.633 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.625+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e33bdfd070 con 0x55e33bed7530 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.624+0000 7f2cf5600700 10 client.26006 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2cf5600700 1 -- 172.21.15.50:0/3404153603 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e33bdfd070 con 0x55e33bed7530 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2cf5600700 10 client.26006 handle_command_reply: tid=0 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 client.26006 shutdown 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 msgr2=0x55e33beb3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d000a6d90 tx=0x7f2d000aa330 comp rx=0 tx=0).stop 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 shutdown_connections 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e33bed7530 0x55e33bed99b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e33be36bf0 0x55e33beb3c50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e33be30c50 0x55e33be4e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 --2- 172.21.15.50:0/3404153603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e33be34a30 0x55e33be4e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2cf5600700 0 client.26006 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.627+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 >> 172.21.15.50:0/3404153603 conn(0x55e33bcf1a80 msgr2=0x55e33be2ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.628+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 shutdown_connections 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.628+0000 7f2d1caabb80 1 -- 172.21.15.50:0/3404153603 wait complete. 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.628+0000 7f2d1caabb80 20 client.26006 trim_cache size 0 max 16384 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.629+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 msgr2=0x7f2cec0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.629+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 0x7f2cec0b5060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0005fd0 tx=0x7f2cf001b440 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.629+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 msgr2=0x7f2d08075230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.629+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b4a0 tx=0x7f2d0000b860 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 shutdown_connections 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2cec0b2b50 0x7f2cec0b5060 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d08110970 0x7f2d08075230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0810a990 0x7f2d081176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 --2- 172.21.15.50:0/1274294478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d0810b2e0 0x7f2d081137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:30.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 >> 172.21.15.50:0/1274294478 conn(0x7f2d08077c80 msgr2=0x7f2d0807bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:30.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 shutdown_connections 2024-05-10T21:11:30.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:30.630+0000 7f2d0e000700 1 -- 172.21.15.50:0/1274294478 wait complete. 2024-05-10T21:11:30.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:30 smithi050 ceph-mon[29546]: pgmap v540: 97 pgs: 97 active+clean; 18 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-05-10T21:11:30.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:30 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:31.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:30 smithi092 ceph-mon[34415]: pgmap v540: 97 pgs: 97 active+clean; 18 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-05-10T21:11:31.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:30 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:31.375 INFO:tasks.workunit.client.0.smithi050.stdout: 13 121327 722 93200 749 17818 506 2024-05-10T21:11:31.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:30 smithi142 ceph-mon[34846]: pgmap v540: 97 pgs: 97 active+clean; 18 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-05-10T21:11:31.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:30 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:33.007 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:11:33.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:11:33.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:32 smithi092 ceph-mon[34415]: pgmap v541: 97 pgs: 97 active+clean; 22 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-05-10T21:11:33.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:32 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:32 smithi142 ceph-mon[34846]: pgmap v541: 97 pgs: 97 active+clean; 22 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-05-10T21:11:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:32 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:33.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:32 smithi050 ceph-mon[29546]: pgmap v541: 97 pgs: 97 active+clean; 22 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-05-10T21:11:33.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:32 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:33.523 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:35.302 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:34 smithi050 ceph-mon[29546]: pgmap v542: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-05-10T21:11:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:34 smithi092 ceph-mon[34415]: pgmap v542: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-05-10T21:11:35.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:34 smithi142 ceph-mon[34846]: pgmap v542: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-05-10T21:11:36.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:35.998+0000 7fe0e4200700 1 -- 172.21.15.50:0/1777692386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 msgr2=0x7fe0dc1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:35.998+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1777692386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1080d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00098f0 tx=0x7fe0d0009c00 comp rx=0 tx=0).stop 2024-05-10T21:11:36.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 -- 172.21.15.50:0/1777692386 shutdown_connections 2024-05-10T21:11:36.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1777692386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc103250 0x7fe0dc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1777692386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc1086a0 0x7fe0dc108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1777692386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1080d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 -- 172.21.15.50:0/1777692386 >> 172.21.15.50:0/1777692386 conn(0x7fe0dc0fedc0 msgr2=0x7fe0dc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 -- 172.21.15.50:0/1777692386 shutdown_connections 2024-05-10T21:11:36.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.003+0000 7fe0e4200700 1 -- 172.21.15.50:0/1777692386 wait complete. 2024-05-10T21:11:36.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.005+0000 7fe0e4200700 1 Processor -- start 2024-05-10T21:11:36.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.009+0000 7fe0e4200700 1 -- start start 2024-05-10T21:11:36.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.009+0000 7fe0e4200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e4200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e4200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc1086a0 0x7fe0dc1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e4200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc11ab60 con 0x7fe0dc103ba0 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e4200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc11a9e0 con 0x7fe0dc1086a0 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e4200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc11ace0 con 0x7fe0dc103250 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41520/0 (socket says 172.21.15.50:41520) 2024-05-10T21:11:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.010+0000 7fe0e1800700 1 -- 172.21.15.50:0/1188751262 learned_addr learned my addr 172.21.15.50:0/1188751262 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e2200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc1086a0 0x7fe0dc1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e1800700 1 -- 172.21.15.50:0/1188751262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc1086a0 msgr2=0x7fe0dc1a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e1800700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc1086a0 0x7fe0dc1a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e1800700 1 -- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 msgr2=0x7fe0dc1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e1800700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.011+0000 7fe0e1800700 1 -- 172.21.15.50:0/1188751262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a62f0 con 0x7fe0dc103250 2024-05-10T21:11:36.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.012+0000 7fe0e1800700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9c0 tx=0x7fe0c800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.012+0000 7fe0da000700 1 -- 172.21.15.50:0/1188751262 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800cbd0 con 0x7fe0dc103250 2024-05-10T21:11:36.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.013+0000 7fe0da000700 1 -- 172.21.15.50:0/1188751262 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c800cd70 con 0x7fe0dc103250 2024-05-10T21:11:36.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.013+0000 7fe0da000700 1 -- 172.21.15.50:0/1188751262 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005580 con 0x7fe0dc103250 2024-05-10T21:11:36.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.015+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a65e0 con 0x7fe0dc103250 2024-05-10T21:11:36.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.015+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1adf00 con 0x7fe0dc103250 2024-05-10T21:11:36.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.025+0000 7fe0f2cb0b80 1 Processor -- start 2024-05-10T21:11:36.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.027+0000 7fe0da000700 1 -- 172.21.15.50:0/1188751262 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe0c8004750 con 0x7fe0dc103250 2024-05-10T21:11:36.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.029+0000 7fe0da000700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 0x7fe0c40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.032+0000 7fe0f2cb0b80 1 -- start start 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 0x560e7d5750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560e7d496aa0 con 0x560e7d3f6ec0 2024-05-10T21:11:36.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e7d4347e0 con 0x560e7d574cd0 2024-05-10T21:11:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.033+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560e7d492a60 con 0x560e7d57ac70 2024-05-10T21:11:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.034+0000 7fe0e0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.034+0000 7fe0e0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41524/0 (socket says 172.21.15.50:41524) 2024-05-10T21:11:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.034+0000 7fe0e0e00700 1 -- 172.21.15.50:0/590117199 learned_addr learned my addr 172.21.15.50:0/590117199 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:36.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.034+0000 7fe0e1800700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.035+0000 7fe0e2200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 0x7fe0c40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.036+0000 7fe0e2200700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 0x560e7d5750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.037+0000 7fe0da000700 1 -- 172.21.15.50:0/1188751262 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe0c80e24f0 con 0x7fe0dc103250 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.041+0000 7fe0e0e00700 1 -- 172.21.15.50:0/590117199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 msgr2=0x560e7d5750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.041+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 0x560e7d5750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.041+0000 7fe0e0e00700 1 -- 172.21.15.50:0/590117199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 msgr2=0x560e7d57a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.041+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.042+0000 7fe0e0e00700 1 -- 172.21.15.50:0/590117199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7d3e3950 con 0x560e7d57ac70 2024-05-10T21:11:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.046+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009230 tx=0x7fe0d000fb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa7d07b0633e583 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.046+0000 7fe0e2200700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 0x560e7d5750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.046+0000 7fe0e1800700 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.046+0000 7fe0cea00700 1 -- 172.21.15.50:0/590117199 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0027070 con 0x560e7d57ac70 2024-05-10T21:11:36.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.047+0000 7fe0cea00700 1 -- 172.21.15.50:0/590117199 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d000f910 con 0x560e7d57ac70 2024-05-10T21:11:36.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.048+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/590117199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 msgr2=0x560e7d574790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.048+0000 7fe0e2200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 0x7fe0c40b4e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400f6e0 tx=0x7fe0d4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.048+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009230 tx=0x7fe0d000fb90 comp rx=0 tx=0).stop 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.049+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/590117199 shutdown_connections 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.049+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d574cd0 0x560e7d5750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.049+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.049+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/590117199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d57ac70 0x560e7d574790 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009230 tx=0x7fe0d000fb90 comp rx=0 tx=0).stop 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.049+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/590117199 >> 172.21.15.50:0/590117199 conn(0x560e7d435b00 msgr2=0x560e7d56f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.052+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/590117199 shutdown_connections 2024-05-10T21:11:36.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.055+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/590117199 wait complete. 2024-05-10T21:11:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.055+0000 7fe0f2cb0b80 1 -- start start 2024-05-10T21:11:36.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.056+0000 7fe0f2cb0b80 1 Processor -- start 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.060+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.061+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.061+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d57ef90 0x560e7d59bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.061+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560e7d5926f0 con 0x560e7d3f6ec0 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.061+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e7d3fcba0 con 0x560e7d57ef90 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560e7d592870 con 0x560e7d574cd0 2024-05-10T21:11:36.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41540/0 (socket says 172.21.15.50:41540) 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 -- 172.21.15.50:0/3805373022 learned_addr learned my addr 172.21.15.50:0/3805373022 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0f2cb0b80 10 client.0 _ll_register_callbacks cb 0x560e7d42a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.061+0000 7fe0e2200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d57ef90 0x560e7d59bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 -- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d57ef90 msgr2=0x560e7d59bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d57ef90 0x560e7d59bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 -- 172.21.15.50:0/3805373022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 msgr2=0x560e7d57e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e1800700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.062+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.063+0000 7fe0e0e00700 1 -- 172.21.15.50:0/3805373022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7d45c4a0 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.063+0000 7fe0e1800700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.065+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0000c00 tx=0x7fe0d000fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c964b948c1f24fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.066+0000 7fe0c3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.066+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0032060 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.066+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe0d000b760 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.066+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e7d5410f0 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.067+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0d0027040 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.067+0000 7fe0f2cb0b80 10 client.25975 fetch_fsmap learned FSMap version 30 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.067+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e7d5926f0 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.067+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560e7d5410f0 con 0x560e7d574cd0 2024-05-10T21:11:36.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.068+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe0d0021070 con 0x560e7d574cd0 2024-05-10T21:11:36.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.068+0000 7fe0f2cb0b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T21:11:36.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.068+0000 7fe0f2cb0b80 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:36.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.068+0000 7fe0f2cb0b80 20 client.25975 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:36.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.068+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d59fbc0 0x560e7d5a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.069+0000 7fe0f2cb0b80 4 client.25975 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.069+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e7d4a4480 con 0x560e7d59fbc0 2024-05-10T21:11:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.069+0000 7fe0e1800700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d59fbc0 0x560e7d5a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.070+0000 7fe0e1800700 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d59fbc0 0x560e7d5a1fe0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379924f2702d81ed server_cookie=4bd9387214adfb5a in_seq=0 out_seq=0 2024-05-10T21:11:36.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.070+0000 7fe0c3400700 10 client.25975 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.071+0000 7fe0c3400700 1 -- 172.21.15.50:0/3805373022 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e7d4a4480 con 0x560e7d59fbc0 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.071+0000 7fe0c3400700 10 client.25975 handle_command_reply: tid=0 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.071+0000 7fe0f2cb0b80 1 client.25975 shutdown 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.073+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 msgr2=0x560e7d57ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.073+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0000c00 tx=0x7fe0d000fa50 comp rx=0 tx=0).stop 2024-05-10T21:11:36.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 shutdown_connections 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d59fbc0 0x560e7d5a1fe0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d57ef90 0x560e7d59bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d3f6ec0 0x560e7d57e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/3805373022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d574cd0 0x560e7d57ea50 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0000c00 tx=0x7fe0d000fa50 comp rx=0 tx=0).stop 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 >> 172.21.15.50:0/3805373022 conn(0x560e7d435b00 msgr2=0x560e7d573780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.074+0000 7fe0c3400700 0 client.25975 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.075+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 shutdown_connections 2024-05-10T21:11:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.075+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/3805373022 wait complete. 2024-05-10T21:11:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.075+0000 7fe0f2cb0b80 20 client.25975 trim_cache size 0 max 16384 2024-05-10T21:11:36.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.124+0000 7fe0f2cb0b80 1 Processor -- start 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.124+0000 7fe0f2cb0b80 1 -- start start 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d578b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1af1b0 con 0x560e7d57ef90 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0742d0 con 0x560e7d3f6ec0 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc072140 con 0x560e7d5a75e0 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.125+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43788/0 (socket says 172.21.15.50:43788) 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 -- 172.21.15.50:0/2777412335 learned_addr learned my addr 172.21.15.50:0/2777412335 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d578b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 -- 172.21.15.50:0/2777412335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 msgr2=0x560e7d5a79f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5a79f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 -- 172.21.15.50:0/2777412335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 msgr2=0x560e7d578b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d578b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e1800700 1 -- 172.21.15.50:0/2777412335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7d569050 con 0x560e7d3f6ec0 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.126+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d578b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.127+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80a5b60 tx=0x7fe0c80e2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64fdb3838433de59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.128+0000 7fe0ce000700 1 -- 172.21.15.50:0/2777412335 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80a9430 con 0x560e7d3f6ec0 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.128+0000 7fe0ce000700 1 -- 172.21.15.50:0/2777412335 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c8005720 con 0x560e7d3f6ec0 2024-05-10T21:11:36.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.128+0000 7fe0ce000700 1 -- 172.21.15.50:0/2777412335 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80a5270 con 0x560e7d3f6ec0 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2777412335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 msgr2=0x560e7d574790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80a5b60 tx=0x7fe0c80e2040 comp rx=0 tx=0).stop 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2777412335 shutdown_connections 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d574790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2777412335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d578b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.129+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2777412335 >> 172.21.15.50:0/2777412335 conn(0x560e7d435b00 msgr2=0x560e7d56f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.130+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2777412335 shutdown_connections 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.130+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2777412335 wait complete. 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.130+0000 7fe0f2cb0b80 1 -- start start 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.131+0000 7fe0f2cb0b80 1 Processor -- start 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.132+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.133+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d597f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.133+0000 7fe0f2cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.133+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc072790 con 0x560e7d57ef90 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc073c00 con 0x560e7d3f6ec0 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0f2cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc072e60 con 0x560e7d5a75e0 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0e1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43790/0 (socket says 172.21.15.50:43790) 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0e1800700 1 -- 172.21.15.50:0/2028246803 learned_addr learned my addr 172.21.15.50:0/2028246803 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:36.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.134+0000 7fe0f2cb0b80 10 client.0 _ll_register_callbacks cb 0x560e7d42a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.135+0000 7fe0e2200700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.137+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d597f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e2200700 1 -- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 msgr2=0x560e7d597a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e2200700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e2200700 1 -- 172.21.15.50:0/2028246803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 msgr2=0x560e7d597f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e2200700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d597f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e2200700 1 -- 172.21.15.50:0/2028246803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7d5947e0 con 0x560e7d5a75e0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.138+0000 7fe0e0e00700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d597f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.140+0000 7fe0c3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.140+0000 7fe0e2200700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5940f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400f460 tx=0x7fe0d4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb4c541e1322c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.141+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4029030 con 0x560e7d5a75e0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.141+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe0d4003a00 con 0x560e7d5a75e0 2024-05-10T21:11:36.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.141+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e7d49e520 con 0x560e7d5a75e0 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.142+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0d4020040 con 0x560e7d5a75e0 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.142+0000 7fe0f2cb0b80 10 client.25978 fetch_fsmap learned FSMap version 30 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.142+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e7d60ab60 con 0x560e7d5a75e0 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.142+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560e7d49e520 con 0x560e7d5a75e0 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.142+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe0d402e880 con 0x560e7d5a75e0 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 20 client.25978 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d60b350 0x560e7d60d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 4 client.25978 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e7d3162d0 con 0x560e7d60b350 2024-05-10T21:11:36.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.143+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d60b350 0x560e7d60d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:36.158 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:11:36.159 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:11:36.159 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "5a9ad03a-a6dd-4dbc-b988-ebaf6b3beebd", 2024-05-10T21:11:36.159 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:11:36.159 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.144+0000 7fe0e1800700 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d60b350 0x560e7d60d770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64554474a2af1656 server_cookie=514a8ba42502a363 in_seq=0 out_seq=0 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.144+0000 7fe0c3400700 10 client.25978 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.145+0000 7fe0c3400700 1 -- 172.21.15.50:0/2028246803 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e7d3162d0 con 0x560e7d60b350 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.145+0000 7fe0c3400700 10 client.25978 handle_command_reply: tid=0 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.145+0000 7fe0f2cb0b80 1 client.25978 shutdown 2024-05-10T21:11:36.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 msgr2=0x560e7d5940f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5940f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400f460 tx=0x7fe0d4004ad0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 shutdown_connections 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560e7d60b350 0x560e7d60d770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e7d3f6ec0 0x560e7d597a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e7d5a75e0 0x560e7d5940f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 --2- 172.21.15.50:0/2028246803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e7d57ef90 0x560e7d597f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0c3400700 0 client.25978 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.146+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 >> 172.21.15.50:0/2028246803 conn(0x560e7d435b00 msgr2=0x560e7d56f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.147+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 shutdown_connections 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.147+0000 7fe0f2cb0b80 1 -- 172.21.15.50:0/2028246803 wait complete. 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.147+0000 7fe0f2cb0b80 20 client.25978 trim_cache size 0 max 16384 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.152+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 msgr2=0x7fe0c40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.152+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 0x7fe0c40b4e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400f6e0 tx=0x7fe0d4009920 comp rx=0 tx=0).stop 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.152+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 msgr2=0x7fe0dc1a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:36.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.152+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9c0 tx=0x7fe0c800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 shutdown_connections 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe0c40b2940 0x7fe0c40b4e50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0dc1086a0 0x7fe0dc1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc103250 0x7fe0dc1a1470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 --2- 172.21.15.50:0/1188751262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0dc103ba0 0x7fe0dc1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 >> 172.21.15.50:0/1188751262 conn(0x7fe0dc0fedc0 msgr2=0x7fe0dc1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 shutdown_connections 2024-05-10T21:11:36.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:36.154+0000 7fe0e4200700 1 -- 172.21.15.50:0/1188751262 wait complete. 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: pgmap v543: 97 pgs: 97 active+clean; 36 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:37.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:36 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:37.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: pgmap v543: 97 pgs: 97 active+clean; 36 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-05-10T21:11:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:11:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:36 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: pgmap v543: 97 pgs: 97 active+clean; 36 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:37.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:36 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:38.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.384 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.385 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.386 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.387 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.389 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:38.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:38.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:38.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: pgmap v544: 97 pgs: 97 active+clean; 43 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-05-10T21:11:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.389 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.390 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.391 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.392 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.395 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.397 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.398 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.399 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:11:39.400 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:38 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: pgmap v544: 97 pgs: 97 active+clean; 43 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-05-10T21:11:39.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.415 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.416 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.418 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:11:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:38 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: pgmap v544: 97 pgs: 97 active+clean; 43 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.421 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.422 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.424 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.425 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.426 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.427 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.428 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.429 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.431 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.434 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.434 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:39.434 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:11:39.434 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:38 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:40.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:39 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:40.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:39 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:40.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:39 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:11:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 -- 172.21.15.50:0/1595029331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac1070c0 msgr2=0x7f4aac1074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/1595029331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac1070c0 0x7f4aac1074d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a9802a3f0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 -- 172.21.15.50:0/1595029331 shutdown_connections 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/1595029331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac10bdc0 0x7f4aac10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/1595029331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac1070c0 0x7f4aac1074d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/1595029331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 -- 172.21.15.50:0/1595029331 >> 172.21.15.50:0/1595029331 conn(0x7f4aac101070 msgr2=0x7f4aac103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 -- 172.21.15.50:0/1595029331 shutdown_connections 2024-05-10T21:11:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.688+0000 7f4ab0a00640 1 -- 172.21.15.50:0/1595029331 wait complete. 2024-05-10T21:11:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.689+0000 7f4ab0a00640 1 Processor -- start 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 -- start start 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac10bdc0 0x7f4aac1a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1119f0 con 0x7f4aac10bdc0 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111870 con 0x7f4aac105e80 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4ab0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111b70 con 0x7f4aac1070c0 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4aa9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4aa9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41574/0 (socket says 172.21.15.50:41574) 2024-05-10T21:11:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.690+0000 7f4aa9400640 1 -- 172.21.15.50:0/58307122 learned_addr learned my addr 172.21.15.50:0/58307122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aaa800640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac10bdc0 0x7f4aac1a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9e00640 1 -- 172.21.15.50:0/58307122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 msgr2=0x7f4aac19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9e00640 1 -- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac10bdc0 msgr2=0x7f4aac1a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac10bdc0 0x7f4aac1a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9e00640 1 -- 172.21.15.50:0/58307122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac1a40e0 con 0x7f4aac105e80 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.691+0000 7f4aa9400640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.692+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac19c5a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4aac105750 tx=0x7f4a9c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.692+0000 7f4aa2a00640 1 -- 172.21.15.50:0/58307122 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00cc40 con 0x7f4aac105e80 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.692+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac1a43d0 con 0x7f4aac105e80 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.692+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a4970 con 0x7f4aac105e80 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.693+0000 7f4aa2a00640 1 -- 172.21.15.50:0/58307122 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9c00cde0 con 0x7f4aac105e80 2024-05-10T21:11:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.693+0000 7f4aa2a00640 1 -- 172.21.15.50:0/58307122 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c010430 con 0x7f4aac105e80 2024-05-10T21:11:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.696+0000 7f4ab7a1e740 1 Processor -- start 2024-05-10T21:11:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 -- start start 2024-05-10T21:11:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be4fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 0x5646be504580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5646be461550 con 0x5646be13a4d0 2024-05-10T21:11:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646be406bc0 con 0x5646be15b1b0 2024-05-10T21:11:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646be45d510 con 0x5646be1322f0 2024-05-10T21:11:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.697+0000 7f4aaa800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43796/0 (socket says 172.21.15.50:43796) 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 -- 172.21.15.50:0/755029031 learned_addr learned my addr 172.21.15.50:0/755029031 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aa9400640 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 0x5646be504580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 -- 172.21.15.50:0/755029031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 msgr2=0x5646be4fd680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be4fd680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 -- 172.21.15.50:0/755029031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 msgr2=0x5646be504580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 0x5646be504580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aaa800640 1 -- 172.21.15.50:0/755029031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646be173210 con 0x5646be15b1b0 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.698+0000 7f4aa9400640 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 0x5646be504580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.699+0000 7f4aaa800640 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009fd0 tx=0x7f4aa40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a7a7f8616ac7642 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.699+0000 7f4aa2a00640 1 -- 172.21.15.50:0/58307122 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4a9c0105d0 con 0x7f4aac105e80 2024-05-10T21:11:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.699+0000 7f4a97000640 1 -- 172.21.15.50:0/755029031 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4004d20 con 0x5646be15b1b0 2024-05-10T21:11:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.699+0000 7f4a97000640 1 -- 172.21.15.50:0/755029031 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa4004050 con 0x5646be15b1b0 2024-05-10T21:11:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.699+0000 7f4a97000640 1 -- 172.21.15.50:0/755029031 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4005240 con 0x5646be15b1b0 2024-05-10T21:11:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/755029031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 msgr2=0x5646be4fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009fd0 tx=0x7f4aa40118d0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/755029031 shutdown_connections 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be4fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be13a4d0 0x5646be504580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/755029031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4fd100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/755029031 >> 172.21.15.50:0/755029031 conn(0x5646be14f300 msgr2=0x5646be4f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/755029031 shutdown_connections 2024-05-10T21:11:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.700+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/755029031 wait complete. 2024-05-10T21:11:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.701+0000 7f4ab7a1e740 1 -- start start 2024-05-10T21:11:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.701+0000 7f4ab7a1e740 1 Processor -- start 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4aa2a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 0x7f4a900b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 0x5646be507230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be50e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5646be504fd0 con 0x5646be15b1b0 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646be41a660 con 0x5646be13a4d0 2024-05-10T21:11:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646be505150 con 0x5646be1322f0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4ab7a1e740 10 client.0 _ll_register_callbacks cb 0x5646be3e4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.702+0000 7f4aa2a00640 1 -- 172.21.15.50:0/58307122 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f4a9c0e2500 con 0x7f4aac105e80 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 0x5646be507230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41582/0 (socket says 172.21.15.50:41582) 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9e00640 1 -- 172.21.15.50:0/885814557 learned_addr learned my addr 172.21.15.50:0/885814557 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aaa800640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be50e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aaa800640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 0x7f4a900b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 -- 172.21.15.50:0/885814557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 msgr2=0x5646be506cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 -- 172.21.15.50:0/885814557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 msgr2=0x5646be50e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be50e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.703+0000 7f4aa9400640 1 -- 172.21.15.50:0/885814557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646be23f620 con 0x5646be13a4d0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.704+0000 7f4aaa800640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be50e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.704+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.704+0000 7f4aa9400640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 0x5646be507230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4aac19d640 tx=0x7f4a980035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4cbc62be95347e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.704+0000 7f4a8b400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.704+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98035040 con 0x5646be13a4d0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4004050 con 0x5646be13a4d0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4aaa800640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 0x7f4a900b4c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009de0 tx=0x7f4aa4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f4a98003aa0 con 0x5646be13a4d0 2024-05-10T21:11:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a9804c070 con 0x5646be13a4d0 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4ab7a1e740 10 client.26033 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.705+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5646be511180 con 0x5646be13a4d0 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa4004050 con 0x5646be13a4d0 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4a9803d450 con 0x5646be13a4d0 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 10 client.26033 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 20 client.26033 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be119d80 0x5646be5148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 4 client.26033 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646be46f160 con 0x5646be119d80 2024-05-10T21:11:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.706+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be119d80 0x5646be5148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.707+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be119d80 0x5646be5148b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3846da48891af8 server_cookie=4f1a76c9e8710c21 in_seq=0 out_seq=0 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.707+0000 7f4a8b400640 10 client.26033 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4a8b400640 1 -- 172.21.15.50:0/885814557 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646be46f160 con 0x5646be119d80 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4a8b400640 10 client.26033 handle_command_reply: tid=0 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4ab7a1e740 1 client.26033 shutdown 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 msgr2=0x5646be507230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 0x5646be507230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4aac19d640 tx=0x7f4a980035f0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.708+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 shutdown_connections 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be119d80 0x5646be5148b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be50e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be13a4d0 0x5646be507230 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/885814557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be1322f0 0x5646be506cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4a8b400640 0 client.26033 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 >> 172.21.15.50:0/885814557 conn(0x5646be14f300 msgr2=0x5646be4f7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 shutdown_connections 2024-05-10T21:11:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/885814557 wait complete. 2024-05-10T21:11:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.709+0000 7f4ab7a1e740 20 client.26033 trim_cache size 0 max 16384 2024-05-10T21:11:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 Processor -- start 2024-05-10T21:11:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 -- start start 2024-05-10T21:11:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c010cb0 con 0x5646be1322f0 2024-05-10T21:11:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00e4c0 con 0x5646be15b1b0 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00c310 con 0x5646be13a4d0 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42884/0 (socket says 172.21.15.50:42884) 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.730+0000 7f4aa9e00640 1 -- 172.21.15.50:0/934965169 learned_addr learned my addr 172.21.15.50:0/934965169 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9400640 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aaa800640 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 -- 172.21.15.50:0/934965169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 msgr2=0x5646be50e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 -- 172.21.15.50:0/934965169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 msgr2=0x5646be4f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 -- 172.21.15.50:0/934965169 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646be4f4280 con 0x5646be1322f0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.731+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c003a40 tx=0x7f4a9c0a5a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12e5613ed73e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.732+0000 7f4a96600640 1 -- 172.21.15.50:0/934965169 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0c3070 con 0x5646be1322f0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.732+0000 7f4a96600640 1 -- 172.21.15.50:0/934965169 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9c0a9af0 con 0x5646be1322f0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.732+0000 7f4a96600640 1 -- 172.21.15.50:0/934965169 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0e00e0 con 0x5646be1322f0 2024-05-10T21:11:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/934965169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 msgr2=0x5646be5066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c003a40 tx=0x7f4a9c0a5a30 comp rx=0 tx=0).stop 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/934965169 shutdown_connections 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be15b1b0 0x5646be4f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/934965169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be1322f0 0x5646be5066b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/934965169 >> 172.21.15.50:0/934965169 conn(0x5646be14f300 msgr2=0x5646be505fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/934965169 shutdown_connections 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.733+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/934965169 wait complete. 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 -- start start 2024-05-10T21:11:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 Processor -- start 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be57abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00c980 con 0x5646be15b1b0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00ddf0 con 0x5646be1322f0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00d050 con 0x5646be13a4d0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.734+0000 7f4ab7a1e740 10 client.0 _ll_register_callbacks cb 0x7f4aa40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43826/0 (socket says 172.21.15.50:43826) 2024-05-10T21:11:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9e00640 1 -- 172.21.15.50:0/2640861066 learned_addr learned my addr 172.21.15.50:0/2640861066 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41588/0 (socket says 172.21.15.50:41588) 2024-05-10T21:11:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aaa800640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be57abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 -- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 msgr2=0x5646be50b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 -- 172.21.15.50:0/2640861066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 msgr2=0x5646be57abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be57abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9400640 1 -- 172.21.15.50:0/2640861066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646be57b140 con 0x5646be13a4d0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.735+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.736+0000 7f4aa9400640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5646be4f2ee0 tx=0x7f4a9803d680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca6765036aa2648 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.736+0000 7f4a8b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.736+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98035040 con 0x5646be13a4d0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.736+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9c0a9af0 con 0x5646be13a4d0 2024-05-10T21:11:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.736+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f4a980038a0 con 0x5646be13a4d0 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a98005dd0 con 0x5646be13a4d0 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 10 client.25993 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5646be57c420 con 0x5646be13a4d0 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9c0a9af0 con 0x5646be13a4d0 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4a98004770 con 0x5646be13a4d0 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 10 client.25993 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 20 client.25993 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be0ef1c0 0x5646be57ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 4 client.25993 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.737+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646be468fd0 con 0x5646be0ef1c0 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.738+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be0ef1c0 0x5646be57ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.738+0000 7f4aa9e00640 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be0ef1c0 0x5646be57ee70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdf52963fd25947 server_cookie=7cd91ea9317561ce in_seq=0 out_seq=0 2024-05-10T21:11:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.738+0000 7f4a8b400640 10 client.25993 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.755+0000 7f4a8b400640 1 -- 172.21.15.50:0/2640861066 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646be468fd0 con 0x5646be0ef1c0 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.755+0000 7f4a8b400640 10 client.25993 handle_command_reply: tid=0 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 client.25993 shutdown 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 msgr2=0x5646be50b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5646be4f2ee0 tx=0x7f4a9803d680 comp rx=0 tx=0).stop 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 shutdown_connections 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5646be0ef1c0 0x5646be57ee70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646be13a4d0 0x5646be50b690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5646be15b1b0 0x5646be57abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4a8b400640 0 client.25993 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 --2- 172.21.15.50:0/2640861066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646be1322f0 0x5646be50b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 >> 172.21.15.50:0/2640861066 conn(0x5646be14f300 msgr2=0x5646be4fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 shutdown_connections 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 1 -- 172.21.15.50:0/2640861066 wait complete. 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.756+0000 7f4ab7a1e740 20 client.25993 trim_cache size 0 max 16384 2024-05-10T21:11:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:11:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:11:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:11:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:11:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.757+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 msgr2=0x7f4a900b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.757+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 0x7f4a900b4c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009de0 tx=0x7f4aa4005fd0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.757+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 msgr2=0x7f4aac19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.757+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac19c5a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4aac105750 tx=0x7f4a9c00ee50 comp rx=0 tx=0).stop 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 shutdown_connections 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f4a900b2770 0x7f4a900b4c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aac1070c0 0x7f4aac19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac10bdc0 0x7f4aac1a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 --2- 172.21.15.50:0/58307122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aac105e80 0x7f4aac19c5a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 >> 172.21.15.50:0/58307122 conn(0x7f4aac101070 msgr2=0x7f4aac107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 shutdown_connections 2024-05-10T21:11:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:40.758+0000 7f4ab0a00640 1 -- 172.21.15.50:0/58307122 wait complete. 2024-05-10T21:11:41.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:11:41.279 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:40 smithi142 ceph-mon[34846]: pgmap v545: 97 pgs: 97 active+clean; 43 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-05-10T21:11:41.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:40 smithi092 ceph-mon[34415]: pgmap v545: 97 pgs: 97 active+clean; 43 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-05-10T21:11:41.375 INFO:tasks.workunit.client.0.smithi050.stdout: 15 162084 157 141107 69 54418 283 2024-05-10T21:11:41.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:40 smithi050 ceph-mon[29546]: pgmap v545: 97 pgs: 97 active+clean; 43 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-05-10T21:11:41.608 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:11:42.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:42 smithi050 ceph-mon[29546]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 166 op/s 2024-05-10T21:11:42.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:42 smithi092 ceph-mon[34415]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 166 op/s 2024-05-10T21:11:42.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:42 smithi142 ceph-mon[34846]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 166 op/s 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.174+0000 7ff76e600700 1 -- 172.21.15.50:0/1927020133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 msgr2=0x7ff7681080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.174+0000 7ff76e600700 1 --2- 172.21.15.50:0/1927020133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681080d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7580098f0 tx=0x7ff758009c00 comp rx=0 tx=0).stop 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.175+0000 7ff76e600700 1 -- 172.21.15.50:0/1927020133 shutdown_connections 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.175+0000 7ff76e600700 1 --2- 172.21.15.50:0/1927020133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7681086a0 0x7ff768108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.175+0000 7ff76e600700 1 --2- 172.21.15.50:0/1927020133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff768103250 0x7ff768103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.175+0000 7ff76e600700 1 --2- 172.21.15.50:0/1927020133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681080d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.175+0000 7ff76e600700 1 -- 172.21.15.50:0/1927020133 >> 172.21.15.50:0/1927020133 conn(0x7ff7680fedc0 msgr2=0x7ff7681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.176+0000 7ff76e600700 1 -- 172.21.15.50:0/1927020133 shutdown_connections 2024-05-10T21:11:44.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.176+0000 7ff76e600700 1 -- 172.21.15.50:0/1927020133 wait complete. 2024-05-10T21:11:44.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.177+0000 7ff76e600700 1 Processor -- start 2024-05-10T21:11:44.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 -- start start 2024-05-10T21:11:44.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 0x7ff7681a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7681086a0 0x7ff7681a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff76811aa20 con 0x7ff7681086a0 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff76811a8a0 con 0x7ff768103ba0 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.178+0000 7ff76e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff76811aba0 con 0x7ff768103250 2024-05-10T21:11:44.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33650/0 (socket says 172.21.15.50:33650) 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 -- 172.21.15.50:0/2587642238 learned_addr learned my addr 172.21.15.50:0/2587642238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff767400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 0x7ff7681a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff767e00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7681086a0 0x7ff7681a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 -- 172.21.15.50:0/2587642238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 msgr2=0x7ff7681a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 0x7ff7681a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 -- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7681086a0 msgr2=0x7ff7681a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7681086a0 0x7ff7681a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.179+0000 7ff766a00700 1 -- 172.21.15.50:0/2587642238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7681a62e0 con 0x7ff768103ba0 2024-05-10T21:11:44.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.180+0000 7ff767400700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 0x7ff7681a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.180+0000 7ff766a00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff75800f990 tx=0x7ff75800f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.180+0000 7ff75fe00700 1 -- 172.21.15.50:0/2587642238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758027070 con 0x7ff768103ba0 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.181+0000 7ff75fe00700 1 -- 172.21.15.50:0/2587642238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7580055f0 con 0x7ff768103ba0 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.181+0000 7ff75fe00700 1 -- 172.21.15.50:0/2587642238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004930 con 0x7ff768103ba0 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.181+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681a6570 con 0x7ff768103ba0 2024-05-10T21:11:44.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.181+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681a6a00 con 0x7ff768103ba0 2024-05-10T21:11:44.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.187+0000 7ff75fe00700 1 -- 172.21.15.50:0/2587642238 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff75802a070 con 0x7ff768103ba0 2024-05-10T21:11:44.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.188+0000 7ff77d0d1b80 1 Processor -- start 2024-05-10T21:11:44.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.188+0000 7ff77d0d1b80 1 -- start start 2024-05-10T21:11:44.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.189+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 0x556e4ae306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.189+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 0x556e4ae2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.189+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4ae2ac50 0x556e4ae2b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556e4ad4ca20 con 0x556e4ae30bf0 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff767e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 0x556e4ae306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff767e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 0x556e4ae306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42014/0 (socket says 172.21.15.50:42014) 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff767e00700 1 -- 172.21.15.50:0/2568359394 learned_addr learned my addr 172.21.15.50:0/2568359394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:44.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff766a00700 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4ae2ac50 0x556e4ae2b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff75fe00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 0x7ff74c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556e4acea760 con 0x556e4ae2ac50 2024-05-10T21:11:44.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff767400700 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 0x556e4ae2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.190+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e4ad489e0 con 0x556e4acace40 2024-05-10T21:11:44.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.191+0000 7ff75fe00700 1 -- 172.21.15.50:0/2587642238 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff7580e2b50 con 0x7ff768103ba0 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767400700 1 -- 172.21.15.50:0/2568359394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 msgr2=0x556e4ae306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767400700 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 0x556e4ae306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767400700 1 -- 172.21.15.50:0/2568359394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4ae2ac50 msgr2=0x556e4ae2b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767400700 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4ae2ac50 0x556e4ae2b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767e00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 0x7ff74c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.192+0000 7ff767400700 1 -- 172.21.15.50:0/2568359394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4ac998d0 con 0x556e4ae30bf0 2024-05-10T21:11:44.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.193+0000 7ff767400700 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 0x556e4ae2a710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7540114c0 tx=0x7ff7540117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392d59d9bc1744f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.193+0000 7ff75cc00700 1 -- 172.21.15.50:0/2568359394 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754004230 con 0x556e4ae30bf0 2024-05-10T21:11:44.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.193+0000 7ff75cc00700 1 -- 172.21.15.50:0/2568359394 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff754011e50 con 0x556e4ae30bf0 2024-05-10T21:11:44.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.193+0000 7ff75cc00700 1 -- 172.21.15.50:0/2568359394 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754005590 con 0x556e4ae30bf0 2024-05-10T21:11:44.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 msgr2=0x556e4ae2a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff767e00700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 0x7ff74c0b4e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x556e4ae2ba30 tx=0x7ff760004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 0x556e4ae2a710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7540114c0 tx=0x7ff7540117d0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 shutdown_connections 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4ae2ac50 0x556e4ae2b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4acace40 0x556e4ae306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/2568359394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae30bf0 0x556e4ae2a710 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.195+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 >> 172.21.15.50:0/2568359394 conn(0x556e4aceba80 msgr2=0x556e4ae25790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.196+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 shutdown_connections 2024-05-10T21:11:44.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.196+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/2568359394 wait complete. 2024-05-10T21:11:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.197+0000 7ff77d0d1b80 1 -- start start 2024-05-10T21:11:44.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.197+0000 7ff77d0d1b80 1 Processor -- start 2024-05-10T21:11:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556e4ae376c0 con 0x556e4ae2ac50 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556e4acb2b20 con 0x556e4acace40 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e4ae37840 con 0x556e4ae30bf0 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.198+0000 7ff77d0d1b80 10 client.0 _ll_register_callbacks cb 0x556e4ace0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff767e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff767e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42020/0 (socket says 172.21.15.50:42020) 2024-05-10T21:11:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae34390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33666/0 (socket says 172.21.15.50:33666) 2024-05-10T21:11:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff767e00700 1 -- 172.21.15.50:0/1798594390 learned_addr learned my addr 172.21.15.50:0/1798594390 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.199+0000 7ff766a00700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.200+0000 7ff767e00700 1 -- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 msgr2=0x556e4ae34390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.200+0000 7ff767e00700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae34390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.200+0000 7ff767e00700 1 -- 172.21.15.50:0/1798594390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 msgr2=0x556e4ae348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.200+0000 7ff767e00700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.200+0000 7ff767e00700 1 -- 172.21.15.50:0/1798594390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4ad12420 con 0x556e4ae30bf0 2024-05-10T21:11:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.201+0000 7ff767e00700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff760007b10 tx=0x7ff760004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58eaf66df006e79a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.201+0000 7ff751600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.201+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760003e10 con 0x556e4ae30bf0 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.201+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff760019890 con 0x556e4ae30bf0 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.201+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff754011e50 con 0x556e4ae30bf0 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.202+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff7600093b0 con 0x556e4ae30bf0 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.202+0000 7ff77d0d1b80 10 client.26005 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:44.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.202+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e4ae422f0 con 0x556e4ae30bf0 2024-05-10T21:11:44.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.202+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff754011e50 con 0x556e4ae30bf0 2024-05-10T21:11:44.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.203+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff76002f430 con 0x556e4ae30bf0 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 10 client.26005 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 20 client.26005 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4ae448d0 0x556e4ae46d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 4 client.26005 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:44.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e4adf7070 con 0x556e4ae448d0 2024-05-10T21:11:44.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.204+0000 7ff767400700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4ae448d0 0x556e4ae46d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.205+0000 7ff767400700 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4ae448d0 0x556e4ae46d60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e599ac9381bf1ac9 server_cookie=fff08dc71419ead6 in_seq=0 out_seq=0 2024-05-10T21:11:44.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.205+0000 7ff751600700 10 client.26005 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:44.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.206+0000 7ff751600700 1 -- 172.21.15.50:0/1798594390 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e4adf7070 con 0x556e4ae448d0 2024-05-10T21:11:44.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.206+0000 7ff751600700 10 client.26005 handle_command_reply: tid=0 2024-05-10T21:11:44.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.206+0000 7ff77d0d1b80 1 client.26005 shutdown 2024-05-10T21:11:44.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 msgr2=0x556e4ae34e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff760007b10 tx=0x7ff760004ad0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 shutdown_connections 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4ae448d0 0x556e4ae46d60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae34390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff751600700 0 client.26005 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/1798594390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.207+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 >> 172.21.15.50:0/1798594390 conn(0x556e4aceba80 msgr2=0x556e4ae29700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.208+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 shutdown_connections 2024-05-10T21:11:44.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.208+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/1798594390 wait complete. 2024-05-10T21:11:44.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.208+0000 7ff77d0d1b80 20 client.26005 trim_cache size 0 max 16384 2024-05-10T21:11:44.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.255+0000 7ff77d0d1b80 1 Processor -- start 2024-05-10T21:11:44.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 -- start start 2024-05-10T21:11:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b7910 con 0x556e4ae2ac50 2024-05-10T21:11:44.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b50e0 con 0x556e4acace40 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.256+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b2f50 con 0x556e4ae30bf0 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.257+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.257+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33682/0 (socket says 172.21.15.50:33682) 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.257+0000 7ff767400700 1 -- 172.21.15.50:0/92167014 learned_addr learned my addr 172.21.15.50:0/92167014 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.257+0000 7ff767e00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.257+0000 7ff766a00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 -- 172.21.15.50:0/92167014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 msgr2=0x556e4ae2a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 -- 172.21.15.50:0/92167014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 msgr2=0x556e4ae258c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae258c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 -- 172.21.15.50:0/92167014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4ae353a0 con 0x556e4ae30bf0 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff766a00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae258c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767400700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.258+0000 7ff767e00700 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff76002fa10 tx=0x7ff76002f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a576bff398a0790 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.259+0000 7ff753e00700 1 -- 172.21.15.50:0/92167014 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760005be0 con 0x556e4ae30bf0 2024-05-10T21:11:44.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.259+0000 7ff753e00700 1 -- 172.21.15.50:0/92167014 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff760005d80 con 0x556e4ae30bf0 2024-05-10T21:11:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.259+0000 7ff753e00700 1 -- 172.21.15.50:0/92167014 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760038740 con 0x556e4ae30bf0 2024-05-10T21:11:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.260+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/92167014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 msgr2=0x556e4ae34c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.260+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff76002fa10 tx=0x7ff76002f6c0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/92167014 shutdown_connections 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae2a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae34c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/92167014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae258c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/92167014 >> 172.21.15.50:0/92167014 conn(0x556e4aceba80 msgr2=0x556e4ae2fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/92167014 shutdown_connections 2024-05-10T21:11:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.261+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/92167014 wait complete. 2024-05-10T21:11:44.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.262+0000 7ff77d0d1b80 1 -- start start 2024-05-10T21:11:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.262+0000 7ff77d0d1b80 1 Processor -- start 2024-05-10T21:11:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b35a0 con 0x556e4ae2ac50 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b4a10 con 0x556e4acace40 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b3c70 con 0x556e4ae30bf0 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.263+0000 7ff77d0d1b80 10 client.0 _ll_register_callbacks cb 0x556e4ace0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33696/0 (socket says 172.21.15.50:33696) 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 -- 172.21.15.50:0/4231761272 learned_addr learned my addr 172.21.15.50:0/4231761272 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:44.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff766a00700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae3cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767e00700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 -- 172.21.15.50:0/4231761272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 msgr2=0x556e4ae39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae39a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 -- 172.21.15.50:0/4231761272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 msgr2=0x556e4ae3cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae3cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.264+0000 7ff767400700 1 -- 172.21.15.50:0/4231761272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4ae39fa0 con 0x556e4acace40 2024-05-10T21:11:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.265+0000 7ff767400700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff754005920 tx=0x7ff754005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51ab98266a9dc1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.265+0000 7ff751600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:11:44.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.265+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754012e60 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.265+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff760005d80 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.265+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff754003680 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.266+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff75403c050 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.266+0000 7ff77d0d1b80 10 client.26057 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.266+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e4aeaf790 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff760005d80 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff75402c710 con 0x556e4acace40 2024-05-10T21:11:44.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 20 client.26057 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4aeafd10 0x556e4aeb2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 4 client.26057 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.267+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e4ad5a400 con 0x556e4aeafd10 2024-05-10T21:11:44.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.268+0000 7ff766a00700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4aeafd10 0x556e4aeb2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:44.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.269+0000 7ff766a00700 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4aeafd10 0x556e4aeb2130 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a75396ee25db77 server_cookie=d11c9ca1966f522b in_seq=0 out_seq=0 2024-05-10T21:11:44.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.269+0000 7ff751600700 10 client.26057 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:44.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.269+0000 7ff751600700 1 -- 172.21.15.50:0/4231761272 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556e4ad5a400 con 0x556e4aeafd10 2024-05-10T21:11:44.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.269+0000 7ff751600700 10 client.26057 handle_command_reply: tid=0 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.269+0000 7ff77d0d1b80 1 client.26057 shutdown 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 msgr2=0x556e4ae3ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff754005920 tx=0x7ff754005260 comp rx=0 tx=0).stop 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 shutdown_connections 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x556e4aeafd10 0x556e4aeb2130 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e4acace40 0x556e4ae3ca90 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff754005920 tx=0x7ff754005260 comp rx=0 tx=0).stop 2024-05-10T21:11:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e4ae30bf0 0x556e4ae39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 --2- 172.21.15.50:0/4231761272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e4ae2ac50 0x556e4ae3cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 >> 172.21.15.50:0/4231761272 conn(0x556e4aceba80 msgr2=0x556e4ae2fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.270+0000 7ff751600700 0 client.26057 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.271+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 shutdown_connections 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.271+0000 7ff77d0d1b80 1 -- 172.21.15.50:0/4231761272 wait complete. 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.271+0000 7ff77d0d1b80 20 client.26057 trim_cache size 0 max 16384 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stdout: "5a9ad03a-a6dd-4dbc-b988-ebaf6b3beebd": { 2024-05-10T21:11:44.270 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:11:44.271 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "5a9ad03a-a6dd-4dbc-b988-ebaf6b3beebd", 2024-05-10T21:11:44.271 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:11:44.271 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:44.271 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:11:44.271 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 msgr2=0x7ff74c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 0x7ff74c0b4e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x556e4ae2ba30 tx=0x7ff760004b40 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 msgr2=0x7ff7681a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff75800f990 tx=0x7ff75800f9c0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 shutdown_connections 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff74c0b2900 0x7ff74c0b4e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7681086a0 0x7ff7681a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff768103250 0x7ff7681a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 --2- 172.21.15.50:0/2587642238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff768103ba0 0x7ff7681a19b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.272+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 >> 172.21.15.50:0/2587642238 conn(0x7ff7680fedc0 msgr2=0x7ff7681060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:44.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.273+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 shutdown_connections 2024-05-10T21:11:44.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:11:44.273+0000 7ff76e600700 1 -- 172.21.15.50:0/2587642238 wait complete. 2024-05-10T21:11:44.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:44 smithi050 ceph-mon[29546]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-05-10T21:11:45.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:44 smithi092 ceph-mon[34415]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-05-10T21:11:45.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:44 smithi142 ceph-mon[34846]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-05-10T21:11:45.317 INFO:tasks.cephfs.filesystem:scrub status for tag:5a9ad03a-a6dd-4dbc-b988-ebaf6b3beebd - {'path': '/', 'tag': '5a9ad03a-a6dd-4dbc-b988-ebaf6b3beebd', 'options': 'recursive,force'} 2024-05-10T21:11:47.127 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:46 smithi092 ceph-mon[34415]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-05-10T21:11:47.143 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:46 smithi142 ceph-mon[34846]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-05-10T21:11:47.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:46 smithi050 ceph-mon[29546]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-05-10T21:11:49.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:48 smithi092 ceph-mon[34415]: pgmap v549: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-05-10T21:11:49.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:48 smithi142 ceph-mon[34846]: pgmap v549: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-05-10T21:11:49.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:48 smithi050 ceph-mon[29546]: pgmap v549: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-05-10T21:11:50.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:49 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:11:50.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:49 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:11:50.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:49 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:11:50.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:49 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:11:50.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:49 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:11:50.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:49 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:11:51.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:50 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:11:51.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:50 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:11:51.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:50 smithi092 ceph-mon[34415]: pgmap v550: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 397 KiB/s wr, 56 op/s 2024-05-10T21:11:51.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:50 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:11:51.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:50 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:11:51.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:50 smithi142 ceph-mon[34846]: pgmap v550: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 397 KiB/s wr, 56 op/s 2024-05-10T21:11:51.178 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:50 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:11:51.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:50 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:11:51.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:50 smithi050 ceph-mon[29546]: pgmap v550: 97 pgs: 97 active+clean; 45 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 397 KiB/s wr, 56 op/s 2024-05-10T21:11:51.380 INFO:tasks.workunit.client.0.smithi050.stdout: 21 213179 304 135633 222 74881 473 2024-05-10T21:11:53.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:52 smithi092 ceph-mon[34415]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 609 KiB/s wr, 53 op/s 2024-05-10T21:11:53.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:11:53.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:11:53.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:11:53.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:52 smithi142 ceph-mon[34846]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 609 KiB/s wr, 53 op/s 2024-05-10T21:11:53.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:11:53.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:11:53.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:11:53.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:52 smithi050 ceph-mon[29546]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 609 KiB/s wr, 53 op/s 2024-05-10T21:11:53.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:11:53.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:11:53.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:11:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.224+0000 7ffa48a00640 1 -- 172.21.15.50:0/1054592485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410ffe0 msgr2=0x7ffa441103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.224+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1054592485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410ffe0 0x7ffa441103f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009950 tx=0x7ffa3002a480 comp rx=0 tx=0).stop 2024-05-10T21:11:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.228+0000 7ffa48a00640 1 -- 172.21.15.50:0/1054592485 shutdown_connections 2024-05-10T21:11:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.228+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1054592485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410ffe0 0x7ffa441103f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.228+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1054592485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410b2c0 0x7ffa4410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.228+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1054592485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410a470 0x7ffa4410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.228+0000 7ffa48a00640 1 -- 172.21.15.50:0/1054592485 >> 172.21.15.50:0/1054592485 conn(0x7ffa44070460 msgr2=0x7ffa44072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.230+0000 7ffa48a00640 1 -- 172.21.15.50:0/1054592485 shutdown_connections 2024-05-10T21:11:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.230+0000 7ffa48a00640 1 -- 172.21.15.50:0/1054592485 wait complete. 2024-05-10T21:11:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.232+0000 7ffa48a00640 1 Processor -- start 2024-05-10T21:11:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.237+0000 7ffa48a00640 1 -- start start 2024-05-10T21:11:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.238+0000 7ffa48a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.240+0000 7ffa48a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.240+0000 7ffa43400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.240+0000 7ffa43400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48798/0 (socket says 172.21.15.50:48798) 2024-05-10T21:11:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.242+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.242+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46704/0 (socket says 172.21.15.50:46704) 2024-05-10T21:11:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.243+0000 7ffa43400640 1 -- 172.21.15.50:0/1936332538 learned_addr learned my addr 172.21.15.50:0/1936332538 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.243+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410ffe0 0x7ffa441b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.244+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4411e4c0 con 0x7ffa4410b2c0 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.244+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4411e340 con 0x7ffa4410a470 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.244+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4411e640 con 0x7ffa4410ffe0 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.244+0000 7ffa43e00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410ffe0 0x7ffa441b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa43400640 1 -- 172.21.15.50:0/1936332538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410ffe0 msgr2=0x7ffa441b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa43400640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410ffe0 0x7ffa441b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa43400640 1 -- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 msgr2=0x7ffa441a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa43400640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa43400640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441b0be0 con 0x7ffa4410a470 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.245+0000 7ffa42a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:11:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.246+0000 7ffa43400640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00b9e0 tx=0x7ffa3c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.262+0000 7ffa3be00640 1 -- 172.21.15.50:0/1936332538 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c019070 con 0x7ffa4410a470 2024-05-10T21:11:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.262+0000 7ffa3be00640 1 -- 172.21.15.50:0/1936332538 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3c00cba0 con 0x7ffa4410a470 2024-05-10T21:11:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.262+0000 7ffa3be00640 1 -- 172.21.15.50:0/1936332538 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0125a0 con 0x7ffa4410a470 2024-05-10T21:11:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.263+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441b0ed0 con 0x7ffa4410a470 2024-05-10T21:11:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.265+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441b1280 con 0x7ffa4410a470 2024-05-10T21:11:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.280+0000 7ffa50fce740 1 Processor -- start 2024-05-10T21:11:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.280+0000 7ffa3be00640 1 -- 172.21.15.50:0/1936332538 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa3c014440 con 0x7ffa4410a470 2024-05-10T21:11:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.283+0000 7ffa3be00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 0x7ffa280b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.283+0000 7ffa3be00640 1 -- 172.21.15.50:0/1936332538 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ffa3c0e1670 con 0x7ffa4410a470 2024-05-10T21:11:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.285+0000 7ffa43e00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 0x7ffa280b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.286+0000 7ffa50fce740 1 -- start start 2024-05-10T21:11:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.292+0000 7ffa43e00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 0x7ffa280b4e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa30031440 tx=0x7ffa300314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 0x55a10d803330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 0x55a10d8038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 0x55a10d80a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a10d767550 con 0x55a10d4611b0 2024-05-10T21:11:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a10d70cbc0 con 0x55a10d4382f0 2024-05-10T21:11:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.293+0000 7ffa50fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a10d763510 con 0x55a10d4404d0 2024-05-10T21:11:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.299+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 0x55a10d80a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.299+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 0x55a10d80a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33274/0 (socket says 172.21.15.50:33274) 2024-05-10T21:11:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.299+0000 7ffa43e00640 1 -- 172.21.15.50:0/3948589507 learned_addr learned my addr 172.21.15.50:0/3948589507 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.302+0000 7ffa42a00640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 0x55a10d8038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa43400640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 0x55a10d803330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa42a00640 1 -- 172.21.15.50:0/3948589507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 msgr2=0x55a10d80a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa42a00640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 0x55a10d80a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa42a00640 1 -- 172.21.15.50:0/3948589507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 msgr2=0x55a10d803330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa42a00640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 0x55a10d803330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa42a00640 1 -- 172.21.15.50:0/3948589507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10d479210 con 0x55a10d4382f0 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.306+0000 7ffa43400640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 0x55a10d803330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.307+0000 7ffa42a00640 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 0x55a10d8038b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa441aa140 tx=0x7ffa3400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ba863a309e8994 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.311+0000 7ffa2fe00640 1 -- 172.21.15.50:0/3948589507 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400d950 con 0x55a10d4382f0 2024-05-10T21:11:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.311+0000 7ffa2fe00640 1 -- 172.21.15.50:0/3948589507 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa34019070 con 0x55a10d4382f0 2024-05-10T21:11:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.311+0000 7ffa2fe00640 1 -- 172.21.15.50:0/3948589507 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340123e0 con 0x55a10d4382f0 2024-05-10T21:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.316+0000 7ffa50fce740 1 -- 172.21.15.50:0/3948589507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 msgr2=0x55a10d8038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.318+0000 7ffa50fce740 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 0x55a10d8038b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa441aa140 tx=0x7ffa3400bc80 comp rx=0 tx=0).stop 2024-05-10T21:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.320+0000 7ffa50fce740 1 -- 172.21.15.50:0/3948589507 shutdown_connections 2024-05-10T21:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.320+0000 7ffa50fce740 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4404d0 0x55a10d80a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.320+0000 7ffa50fce740 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4382f0 0x55a10d8038b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.320+0000 7ffa50fce740 1 --2- 172.21.15.50:0/3948589507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4611b0 0x55a10d803330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.320+0000 7ffa50fce740 1 -- 172.21.15.50:0/3948589507 >> 172.21.15.50:0/3948589507 conn(0x55a10d455300 msgr2=0x55a10d7fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.324+0000 7ffa50fce740 1 -- 172.21.15.50:0/3948589507 shutdown_connections 2024-05-10T21:11:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.324+0000 7ffa50fce740 1 -- 172.21.15.50:0/3948589507 wait complete. 2024-05-10T21:11:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.325+0000 7ffa50fce740 1 -- start start 2024-05-10T21:11:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.325+0000 7ffa50fce740 1 Processor -- start 2024-05-10T21:11:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.329+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d80cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.332+0000 7ffa43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d80cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.332+0000 7ffa43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d80cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33280/0 (socket says 172.21.15.50:33280) 2024-05-10T21:11:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.332+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46732/0 (socket says 172.21.15.50:46732) 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa43400640 1 -- 172.21.15.50:0/2927933223 learned_addr learned my addr 172.21.15.50:0/2927933223 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d814360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a10d80b200 con 0x55a10d4404d0 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a10d720660 con 0x55a10d4611b0 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.333+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a10d80b380 con 0x55a10d4382f0 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.334+0000 7ffa43e00640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d814360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 -- 172.21.15.50:0/2927933223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 msgr2=0x55a10d80cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d80cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 -- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 msgr2=0x55a10d814360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d814360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10d545620 con 0x55a10d4404d0 2024-05-10T21:11:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.335+0000 7ffa42a00640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55a10d7ffa50 tx=0x7ffa34014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2cae7e4c3798d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.336+0000 7ffa50fce740 10 client.0 _ll_register_callbacks cb 0x55a10d6ea400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.336+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa34019070 con 0x55a10d4404d0 2024-05-10T21:11:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.338+0000 7ffa2cc00640 10 client.15927 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:11:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.338+0000 7ffa2cc00640 1 -- 172.21.15.50:0/2927933223 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34023070 con 0x55a10d4404d0 2024-05-10T21:11:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.338+0000 7ffa2cc00640 1 -- 172.21.15.50:0/2927933223 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ffa3400db50 con 0x55a10d4404d0 2024-05-10T21:11:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.352+0000 7ffa2cc00640 1 -- 172.21.15.50:0/2927933223 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa3402d050 con 0x55a10d4404d0 2024-05-10T21:11:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.365+0000 7ffa50fce740 10 client.15927 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.365+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a10d817020 con 0x55a10d4404d0 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa2cc00640 1 -- 172.21.15.50:0/2927933223 <== mon.0 v2:172.21.15.50:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffa34013070 con 0x55a10d4404d0 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 20 client.15927 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d41fd80 0x55a10d81a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 4 client.15927 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:11:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a10d775160 con 0x55a10d41fd80 2024-05-10T21:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.366+0000 7ffa43400640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d41fd80 0x55a10d81a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.379+0000 7ffa43400640 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d41fd80 0x55a10d81a400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafbe84c14b36aab server_cookie=4387ade6f336c5a4 in_seq=0 out_seq=0 2024-05-10T21:11:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa2cc00640 10 client.15927 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa2cc00640 1 -- 172.21.15.50:0/2927933223 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a10d775160 con 0x55a10d41fd80 2024-05-10T21:11:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa2cc00640 10 client.15927 handle_command_reply: tid=0 2024-05-10T21:11:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa50fce740 1 client.15927 shutdown 2024-05-10T21:11:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 msgr2=0x55a10d80d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.384+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55a10d7ffa50 tx=0x7ffa34014660 comp rx=0 tx=0).stop 2024-05-10T21:11:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 shutdown_connections 2024-05-10T21:11:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d41fd80 0x55a10d81a400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d814360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d80d400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2927933223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d80cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 >> 172.21.15.50:0/2927933223 conn(0x55a10d455300 msgr2=0x55a10d801bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.385+0000 7ffa2cc00640 0 client.15927 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.390+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 shutdown_connections 2024-05-10T21:11:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.394+0000 7ffa50fce740 1 -- 172.21.15.50:0/2927933223 wait complete. 2024-05-10T21:11:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.403+0000 7ffa50fce740 20 client.15927 trim_cache size 0 max 16384 2024-05-10T21:11:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.561+0000 7ffa50fce740 1 Processor -- start 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.561+0000 7ffa50fce740 1 -- start start 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4382f0 0x55a10d7feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4404d0 0x55a10d814280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441bc930 con 0x55a10d4382f0 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441ba140 con 0x55a10d4404d0 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa50fce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441b7f90 con 0x55a10d4611b0 2024-05-10T21:11:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33288/0 (socket says 172.21.15.50:33288) 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.562+0000 7ffa43e00640 1 -- 172.21.15.50:0/2284151882 learned_addr learned my addr 172.21.15.50:0/2284151882 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 -- 172.21.15.50:0/2284151882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4404d0 msgr2=0x55a10d814280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4404d0 0x55a10d814280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 -- 172.21.15.50:0/2284151882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4382f0 msgr2=0x55a10d7feb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4382f0 0x55a10d7feb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 -- 172.21.15.50:0/2284151882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10d41ffb0 con 0x55a10d4611b0 2024-05-10T21:11:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.563+0000 7ffa43e00640 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009950 tx=0x7ffa3003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8089580d71c610 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.566+0000 7ffa2f400640 1 -- 172.21.15.50:0/2284151882 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30035050 con 0x55a10d4611b0 2024-05-10T21:11:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.566+0000 7ffa2f400640 1 -- 172.21.15.50:0/2284151882 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa30003c20 con 0x55a10d4611b0 2024-05-10T21:11:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.569+0000 7ffa2f400640 1 -- 172.21.15.50:0/2284151882 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3003cbd0 con 0x55a10d4611b0 2024-05-10T21:11:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.579+0000 7ffa50fce740 1 -- 172.21.15.50:0/2284151882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 msgr2=0x55a10d7fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.579+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009950 tx=0x7ffa3003e000 comp rx=0 tx=0).stop 2024-05-10T21:11:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.581+0000 7ffa50fce740 1 -- 172.21.15.50:0/2284151882 shutdown_connections 2024-05-10T21:11:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.581+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4611b0 0x55a10d7fd250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.581+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4404d0 0x55a10d814280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.581+0000 7ffa50fce740 1 --2- 172.21.15.50:0/2284151882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4382f0 0x55a10d7feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.581+0000 7ffa50fce740 1 -- 172.21.15.50:0/2284151882 >> 172.21.15.50:0/2284151882 conn(0x55a10d455300 msgr2=0x55a10d7fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.584+0000 7ffa50fce740 1 -- 172.21.15.50:0/2284151882 shutdown_connections 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.584+0000 7ffa50fce740 1 -- 172.21.15.50:0/2284151882 wait complete. 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.584+0000 7ffa50fce740 1 -- start start 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.585+0000 7ffa50fce740 1 Processor -- start 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.598+0000 7ffa50fce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.601+0000 7ffa43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.601+0000 7ffa43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33290/0 (socket says 172.21.15.50:33290) 2024-05-10T21:11:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.602+0000 7ffa43400640 1 -- 172.21.15.50:0/1018305726 learned_addr learned my addr 172.21.15.50:0/1018305726 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.602+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d811920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.603+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d812850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.603+0000 7ffa42a00640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d811920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.604+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441b8600 con 0x55a10d4404d0 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.604+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441b9a70 con 0x55a10d4611b0 2024-05-10T21:11:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.604+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441b8cd0 con 0x55a10d4382f0 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.605+0000 7ffa43400640 1 -- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 msgr2=0x55a10d812850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.604+0000 7ffa43e00640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d812850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.606+0000 7ffa43400640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d812850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.606+0000 7ffa43400640 1 -- 172.21.15.50:0/1018305726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 msgr2=0x55a10d811920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.606+0000 7ffa43400640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d811920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.606+0000 7ffa43400640 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10d812e10 con 0x55a10d4382f0 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.607+0000 7ffa43400640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0a8a70 tx=0x7ffa3c004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fa72d8a52b57cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.607+0000 7ffa43e00640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d812850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:11:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.608+0000 7ffa50fce740 10 client.0 _ll_register_callbacks cb 0x7ffa30004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:11:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.608+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa30003c20 con 0x55a10d4382f0 2024-05-10T21:11:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa2cc00640 10 client.26026 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:11:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa2cc00640 1 -- 172.21.15.50:0/1018305726 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0c8060 con 0x55a10d4382f0 2024-05-10T21:11:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa2cc00640 1 -- 172.21.15.50:0/1018305726 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ffa3c0c32c0 con 0x55a10d4382f0 2024-05-10T21:11:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa2cc00640 1 -- 172.21.15.50:0/1018305726 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c0a5000 con 0x55a10d4382f0 2024-05-10T21:11:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa50fce740 10 client.26026 fetch_fsmap learned FSMap version 31 2024-05-10T21:11:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.609+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a10d813ed0 con 0x55a10d4382f0 2024-05-10T21:11:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.611+0000 7ffa2cc00640 1 -- 172.21.15.50:0/1018305726 <== mon.1 v2:172.21.15.142:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffa3c0b1690 con 0x55a10d4382f0 2024-05-10T21:11:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 10 client.26026 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:11:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:11:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 20 client.26026 populate_metadata read hostname 'smithi050' 2024-05-10T21:11:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d3f51c0 0x55a10d895790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:11:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 4 client.26026 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:11:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ffa30003c20 con 0x55a10d3f51c0 2024-05-10T21:11:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.612+0000 7ffa42a00640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d3f51c0 0x55a10d895790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:11:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.613+0000 7ffa42a00640 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d3f51c0 0x55a10d895790 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f93a7a20f7a01f server_cookie=b70a01af07c6631 in_seq=0 out_seq=0 2024-05-10T21:11:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:54.614+0000 7ffa2cc00640 10 client.26026 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:55.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:54 smithi092 ceph-mon[34415]: pgmap v552: 97 pgs: 97 active+clean; 51 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 709 KiB/s wr, 72 op/s 2024-05-10T21:11:55.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:54 smithi142 ceph-mon[34846]: pgmap v552: 97 pgs: 97 active+clean; 51 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 709 KiB/s wr, 72 op/s 2024-05-10T21:11:55.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:54 smithi050 ceph-mon[29546]: pgmap v552: 97 pgs: 97 active+clean; 51 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 709 KiB/s wr, 72 op/s 2024-05-10T21:11:57.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:56 smithi092 ceph-mon[34415]: pgmap v553: 97 pgs: 97 active+clean; 57 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-10T21:11:57.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:56 smithi142 ceph-mon[34846]: pgmap v553: 97 pgs: 97 active+clean; 57 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-10T21:11:57.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:56 smithi050 ceph-mon[29546]: pgmap v553: 97 pgs: 97 active+clean; 57 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-05-10T21:11:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:11:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:11:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:11:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:11:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.923+0000 7ffa2cc00640 1 -- 172.21.15.50:0/1018305726 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ffa30003c20 con 0x55a10d3f51c0 2024-05-10T21:11:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.923+0000 7ffa2cc00640 10 client.26026 handle_command_reply: tid=0 2024-05-10T21:11:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.923+0000 7ffa50fce740 1 client.26026 shutdown 2024-05-10T21:11:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 msgr2=0x55a10d8113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0a8a70 tx=0x7ffa3c004ac0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 shutdown_connections 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a10d3f51c0 0x55a10d895790 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a10d4611b0 0x55a10d812850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa2cc00640 0 client.26026 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a10d4404d0 0x55a10d811920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 --2- 172.21.15.50:0/1018305726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a10d4382f0 0x55a10d8113e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.924+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 >> 172.21.15.50:0/1018305726 conn(0x55a10d455300 msgr2=0x55a10d7fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.925+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 shutdown_connections 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.925+0000 7ffa50fce740 1 -- 172.21.15.50:0/1018305726 wait complete. 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.925+0000 7ffa50fce740 20 client.26026 trim_cache size 0 max 16384 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.926+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 msgr2=0x7ffa280b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.926+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 0x7ffa280b4e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa30031440 tx=0x7ffa300314a0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 msgr2=0x7ffa441a90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00b9e0 tx=0x7ffa3c00bea0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 shutdown_connections 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ffa280b2930 0x7ffa280b4e40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4410ffe0 0x7ffa441b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.927+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa4410b2c0 0x7ffa441a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.928+0000 7ffa48a00640 1 --2- 172.21.15.50:0/1936332538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4410a470 0x7ffa441a90a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:11:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.928+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 >> 172.21.15.50:0/1936332538 conn(0x7ffa44070460 msgr2=0x7ffa4410bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:11:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.928+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 shutdown_connections 2024-05-10T21:11:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:11:58.928+0000 7ffa48a00640 1 -- 172.21.15.50:0/1936332538 wait complete. 2024-05-10T21:11:59.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:11:58 smithi050 ceph-mon[29546]: pgmap v554: 97 pgs: 97 active+clean; 65 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-05-10T21:11:59.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:11:58 smithi092 ceph-mon[34415]: pgmap v554: 97 pgs: 97 active+clean; 65 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-05-10T21:11:59.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:11:58 smithi142 ceph-mon[34846]: pgmap v554: 97 pgs: 97 active+clean; 65 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-05-10T21:12:01.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:00 smithi050 ceph-mon[29546]: pgmap v555: 97 pgs: 97 active+clean; 70 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-05-10T21:12:01.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:00 smithi092 ceph-mon[34415]: pgmap v555: 97 pgs: 97 active+clean; 70 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-05-10T21:12:01.377 INFO:tasks.workunit.client.0.smithi050.stdout: 29 381304 339 245020 276 140929 436 2024-05-10T21:12:01.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:00 smithi142 ceph-mon[34846]: pgmap v555: 97 pgs: 97 active+clean; 70 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-05-10T21:12:03.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:02 smithi050 ceph-mon[29546]: pgmap v556: 97 pgs: 97 active+clean; 80 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-05-10T21:12:03.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:02 smithi092 ceph-mon[34415]: pgmap v556: 97 pgs: 97 active+clean; 80 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-05-10T21:12:03.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:02 smithi142 ceph-mon[34846]: pgmap v556: 97 pgs: 97 active+clean; 80 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-05-10T21:12:05.197 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:04 smithi050 ceph-mon[29546]: pgmap v557: 97 pgs: 97 active+clean; 85 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-05-10T21:12:05.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:04 smithi092 ceph-mon[34415]: pgmap v557: 97 pgs: 97 active+clean; 85 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-05-10T21:12:05.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:04 smithi142 ceph-mon[34846]: pgmap v557: 97 pgs: 97 active+clean; 85 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 381 op/s 2024-05-10T21:12:07.181 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:06 smithi050 ceph-mon[29546]: pgmap v558: 97 pgs: 97 active+clean; 88 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-05-10T21:12:07.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:06 smithi092 ceph-mon[34415]: pgmap v558: 97 pgs: 97 active+clean; 88 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-05-10T21:12:07.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:06 smithi142 ceph-mon[34846]: pgmap v558: 97 pgs: 97 active+clean; 88 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-05-10T21:12:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:08 smithi050 ceph-mon[29546]: pgmap v559: 97 pgs: 97 active+clean; 88 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 295 op/s 2024-05-10T21:12:09.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:08 smithi092 ceph-mon[34415]: pgmap v559: 97 pgs: 97 active+clean; 88 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 295 op/s 2024-05-10T21:12:09.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:08 smithi142 ceph-mon[34846]: pgmap v559: 97 pgs: 97 active+clean; 88 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 295 op/s 2024-05-10T21:12:11.178 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:10 smithi142 ceph-mon[34846]: pgmap v560: 97 pgs: 97 active+clean; 94 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-05-10T21:12:11.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:10 smithi092 ceph-mon[34415]: pgmap v560: 97 pgs: 97 active+clean; 94 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-05-10T21:12:11.377 INFO:tasks.workunit.client.0.smithi050.stdout: 34 395060 500 246455 315 143840 563 2024-05-10T21:12:11.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:10 smithi050 ceph-mon[29546]: pgmap v560: 97 pgs: 97 active+clean; 94 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-05-10T21:12:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.266+0000 7f0a6da00640 1 -- 172.21.15.50:0/1379558649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600aad00 msgr2=0x7f0a600ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.266+0000 7f0a6da00640 1 --2- 172.21.15.50:0/1379558649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600aad00 0x7f0a600ad120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009900 tx=0x7f0a5402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 -- 172.21.15.50:0/1379558649 shutdown_connections 2024-05-10T21:12:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 --2- 172.21.15.50:0/1379558649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a600afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 --2- 172.21.15.50:0/1379558649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600aad00 0x7f0a600ad120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 --2- 172.21.15.50:0/1379558649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600a5740 0x7f0a600a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 -- 172.21.15.50:0/1379558649 >> 172.21.15.50:0/1379558649 conn(0x7f0a600a09a0 msgr2=0x7f0a600a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 -- 172.21.15.50:0/1379558649 shutdown_connections 2024-05-10T21:12:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.269+0000 7f0a6da00640 1 -- 172.21.15.50:0/1379558649 wait complete. 2024-05-10T21:12:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.271+0000 7f0a6da00640 1 Processor -- start 2024-05-10T21:12:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.272+0000 7f0a6da00640 1 -- start start 2024-05-10T21:12:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.273+0000 7f0a6da00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.273+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.273+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48306/0 (socket says 172.21.15.50:48306) 2024-05-10T21:12:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.273+0000 7f0a66a00640 1 -- 172.21.15.50:0/4050401644 learned_addr learned my addr 172.21.15.50:0/4050401644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.274+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 0x7f0a601408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.274+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a60147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.275+0000 7f0a67400640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a60147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.275+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600a8690 con 0x7f0a600ad6b0 2024-05-10T21:12:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.275+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600a8510 con 0x7f0a600a5740 2024-05-10T21:12:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.275+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600a8810 con 0x7f0a600aad00 2024-05-10T21:12:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66000640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 0x7f0a601408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66a00640 1 -- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 msgr2=0x7f0a601408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66a00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 0x7f0a601408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66a00640 1 -- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 msgr2=0x7f0a60147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66a00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a60147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66a00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a60147e20 con 0x7f0a600a5740 2024-05-10T21:12:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a67400640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a60147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.276+0000 7f0a66000640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 0x7f0a601408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.277+0000 7f0a66a00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a68066580 tx=0x7f0a68071820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.277+0000 7f0a5f400640 1 -- 172.21.15.50:0/4050401644 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68076590 con 0x7f0a600a5740 2024-05-10T21:12:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.277+0000 7f0a5f400640 1 -- 172.21.15.50:0/4050401644 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a68076be0 con 0x7f0a600a5740 2024-05-10T21:12:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.277+0000 7f0a5f400640 1 -- 172.21.15.50:0/4050401644 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68077cd0 con 0x7f0a600a5740 2024-05-10T21:12:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.277+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a60148110 con 0x7f0a600a5740 2024-05-10T21:12:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.278+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a60148660 con 0x7f0a600a5740 2024-05-10T21:12:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.290+0000 7f0a5f400640 1 -- 172.21.15.50:0/4050401644 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0a6807e030 con 0x7f0a600a5740 2024-05-10T21:12:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.292+0000 7f0a74990740 1 Processor -- start 2024-05-10T21:12:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.294+0000 7f0a74990740 1 -- start start 2024-05-10T21:12:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4073f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5580a3fc8550 con 0x5580a3ca14d0 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580a3f6dbc0 con 0x5580a3c992f0 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a74990740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580a3fc4510 con 0x5580a3cc21b0 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a5f400640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 0x7f0a4c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a67400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a67400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a406cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46478/0 (socket says 172.21.15.50:46478) 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a67400640 1 -- 172.21.15.50:0/3891208331 learned_addr learned my addr 172.21.15.50:0/3891208331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.295+0000 7f0a66000640 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.296+0000 7f0a5f400640 1 -- 172.21.15.50:0/4050401644 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a68145e80 con 0x7f0a600a5740 2024-05-10T21:12:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a66000640 1 -- 172.21.15.50:0/3891208331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 msgr2=0x5580a406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a66000640 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a66000640 1 -- 172.21.15.50:0/3891208331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 msgr2=0x5580a406d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:12:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a66000640 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a406d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a66000640 1 -- 172.21.15.50:0/3891208331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a3cda210 con 0x5580a3ca14d0 2024-05-10T21:12:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.298+0000 7f0a67400640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 0x7f0a4c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.300+0000 7f0a66000640 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4073f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a5402a9e0 tx=0x7f0a540043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138c159e558f346 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.300+0000 7f0a67400640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 0x7f0a4c0b4d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5580a406dcd0 tx=0x7f0a58004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.301+0000 7f0a53e00640 1 -- 172.21.15.50:0/3891208331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54035030 con 0x5580a3ca14d0 2024-05-10T21:12:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.301+0000 7f0a53e00640 1 -- 172.21.15.50:0/3891208331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54005de0 con 0x5580a3ca14d0 2024-05-10T21:12:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.301+0000 7f0a53e00640 1 -- 172.21.15.50:0/3891208331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54031dd0 con 0x5580a3ca14d0 2024-05-10T21:12:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.302+0000 7f0a74990740 1 -- 172.21.15.50:0/3891208331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 msgr2=0x5580a4073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.302+0000 7f0a74990740 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4073f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a5402a9e0 tx=0x7f0a540043d0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 -- 172.21.15.50:0/3891208331 shutdown_connections 2024-05-10T21:12:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4073f80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 --2- 172.21.15.50:0/3891208331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 -- 172.21.15.50:0/3891208331 >> 172.21.15.50:0/3891208331 conn(0x5580a3cb6300 msgr2=0x5580a405fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 -- 172.21.15.50:0/3891208331 shutdown_connections 2024-05-10T21:12:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.305+0000 7f0a74990740 1 -- 172.21.15.50:0/3891208331 wait complete. 2024-05-10T21:12:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.306+0000 7f0a74990740 1 -- start start 2024-05-10T21:12:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.306+0000 7f0a74990740 1 Processor -- start 2024-05-10T21:12:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.307+0000 7f0a74990740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.307+0000 7f0a74990740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4076b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46482/0 (socket says 172.21.15.50:46482) 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a74990740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a74990740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5580a4074970 con 0x5580a3ca14d0 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a74990740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580a3f81660 con 0x5580a3cc21b0 2024-05-10T21:12:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a74990740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580a4074af0 con 0x5580a3c992f0 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a74990740 10 client.0 _ll_register_callbacks cb 0x5580a3f4b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.308+0000 7f0a66a00640 1 -- 172.21.15.50:0/2924000761 learned_addr learned my addr 172.21.15.50:0/2924000761 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.309+0000 7f0a66000640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.309+0000 7f0a67400640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.309+0000 7f0a66a00640 1 -- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 msgr2=0x5580a407da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.309+0000 7f0a66a00640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.309+0000 7f0a66a00640 1 -- 172.21.15.50:0/2924000761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 msgr2=0x5580a4076b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.310+0000 7f0a66a00640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4076b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.310+0000 7f0a66a00640 1 -- 172.21.15.50:0/2924000761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a3da6620 con 0x5580a3c992f0 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.310+0000 7f0a66000640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.310+0000 7f0a66a00640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a68068bf0 tx=0x7f0a6810d150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0a37e6b5917751 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.310+0000 7f0a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a681162d0 con 0x5580a3c992f0 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a680f60b0 con 0x5580a3c992f0 2024-05-10T21:12:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580a3fd6160 con 0x5580a3c992f0 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a68109790 con 0x5580a3c992f0 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a74990740 10 client.26035 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.311+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580a4080a00 con 0x5580a3c992f0 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580a3fd6160 con 0x5580a3c992f0 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0a6810c880 con 0x5580a3c992f0 2024-05-10T21:12:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a74990740 10 client.26035 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a74990740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a74990740 20 client.26035 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.312+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c80d80 0x5580a4084150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.313+0000 7f0a74990740 4 client.26035 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.313+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580a3fcffd0 con 0x5580a3c80d80 2024-05-10T21:12:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.313+0000 7f0a66000640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c80d80 0x5580a4084150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.314+0000 7f0a66000640 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c80d80 0x5580a4084150 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95e2d56266ed7c7 server_cookie=aa5793d68dfa3b6c in_seq=0 out_seq=0 2024-05-10T21:12:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.314+0000 7f0a50c00640 10 client.26035 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.315+0000 7f0a50c00640 1 -- 172.21.15.50:0/2924000761 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580a3fcffd0 con 0x5580a3c80d80 2024-05-10T21:12:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.315+0000 7f0a50c00640 10 client.26035 handle_command_reply: tid=0 2024-05-10T21:12:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 client.26035 shutdown 2024-05-10T21:12:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 msgr2=0x5580a40765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a68068bf0 tx=0x7f0a6810d150 comp rx=0 tx=0).stop 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 shutdown_connections 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c80d80 0x5580a4084150 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 --2- 172.21.15.50:0/2924000761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3c992f0 0x5580a40765c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 >> 172.21.15.50:0/2924000761 conn(0x5580a3cb6300 msgr2=0x5580a405ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.316+0000 7f0a50c00640 0 client.26035 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.317+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 shutdown_connections 2024-05-10T21:12:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.317+0000 7f0a74990740 1 -- 172.21.15.50:0/2924000761 wait complete. 2024-05-10T21:12:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.317+0000 7f0a74990740 20 client.26035 trim_cache size 0 max 16384 2024-05-10T21:12:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 Processor -- start 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 -- start start 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3c992f0 0x5580a4071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 0x5580a407d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48010cb0 con 0x5580a3c992f0 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4800e4c0 con 0x5580a3cc21b0 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.343+0000 7f0a74990740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4800c310 con 0x5580a3ca14d0 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a67400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 0x5580a407d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a67400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48326/0 (socket says 172.21.15.50:48326) 2024-05-10T21:12:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a67400640 1 -- 172.21.15.50:0/3393658565 learned_addr learned my addr 172.21.15.50:0/3393658565 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 -- 172.21.15.50:0/3393658565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 msgr2=0x5580a407df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 -- 172.21.15.50:0/3393658565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3c992f0 msgr2=0x5580a4071ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3c992f0 0x5580a4071ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.344+0000 7f0a66000640 1 -- 172.21.15.50:0/3393658565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a405b470 con 0x5580a3ca14d0 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.345+0000 7f0a66000640 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 0x5580a407d9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5580a4077630 tx=0x7f0a54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1163be229120f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.345+0000 7f0a53400640 1 -- 172.21.15.50:0/3393658565 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54035040 con 0x5580a3ca14d0 2024-05-10T21:12:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.345+0000 7f0a53400640 1 -- 172.21.15.50:0/3393658565 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54031ce0 con 0x5580a3ca14d0 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.345+0000 7f0a53400640 1 -- 172.21.15.50:0/3393658565 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54052370 con 0x5580a3ca14d0 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 -- 172.21.15.50:0/3393658565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 msgr2=0x5580a407d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 0x5580a407d9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5580a4077630 tx=0x7f0a54009260 comp rx=0 tx=0).stop 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 -- 172.21.15.50:0/3393658565 shutdown_connections 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3ca14d0 0x5580a407d9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3cc21b0 0x5580a407df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 --2- 172.21.15.50:0/3393658565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3c992f0 0x5580a4071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 -- 172.21.15.50:0/3393658565 >> 172.21.15.50:0/3393658565 conn(0x5580a3cb6300 msgr2=0x5580a4063320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 -- 172.21.15.50:0/3393658565 shutdown_connections 2024-05-10T21:12:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.346+0000 7f0a74990740 1 -- 172.21.15.50:0/3393658565 wait complete. 2024-05-10T21:12:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.347+0000 7f0a74990740 1 -- start start 2024-05-10T21:12:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.347+0000 7f0a74990740 1 Processor -- start 2024-05-10T21:12:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.347+0000 7f0a74990740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.347+0000 7f0a74990740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a40e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.347+0000 7f0a74990740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a40ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a74990740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4800c980 con 0x5580a3ca14d0 2024-05-10T21:12:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a74990740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4800ddf0 con 0x5580a3c992f0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a74990740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4800d050 con 0x5580a3cc21b0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a74990740 10 client.0 _ll_register_callbacks cb 0x7f0a681093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:48328/0 (socket says 172.21.15.50:48328) 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 -- 172.21.15.50:0/295325943 learned_addr learned my addr 172.21.15.50:0/295325943 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a67400640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a40ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66000640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a40e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 -- 172.21.15.50:0/295325943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 msgr2=0x5580a40ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a40ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 -- 172.21.15.50:0/295325943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 msgr2=0x5580a40e30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.348+0000 7f0a66a00640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a40e30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.349+0000 7f0a66a00640 1 -- 172.21.15.50:0/295325943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a40ea600 con 0x5580a3c992f0 2024-05-10T21:12:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.350+0000 7f0a66a00640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a680512f0 tx=0x7f0a681160f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61beaedf549c55f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.350+0000 7f0a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.350+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a681162d0 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.350+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a54031ce0 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.352+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a680f60b0 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.352+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a6812c050 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.352+0000 7f0a74990740 10 client.26081 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.352+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580a40eb890 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54031ce0 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0a6810c1d0 con 0x5580a3c992f0 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 10 client.26081 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 20 client.26081 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c561c0 0x5580a40ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 4 client.26081 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:12:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.353+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580a3f4b4b0 con 0x5580a3c561c0 2024-05-10T21:12:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.354+0000 7f0a66000640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c561c0 0x5580a40ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.354+0000 7f0a66000640 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c561c0 0x5580a40ee2e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99747255f22d77cf server_cookie=1d7ac05655e83c65 in_seq=0 out_seq=0 2024-05-10T21:12:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.354+0000 7f0a50c00640 10 client.26081 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.667+0000 7f0a50c00640 1 -- 172.21.15.50:0/295325943 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580a3f4b4b0 con 0x5580a3c561c0 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.667+0000 7f0a50c00640 10 client.26081 handle_command_reply: tid=0 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.667+0000 7f0a74990740 1 client.26081 shutdown 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 msgr2=0x5580a40e2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a680512f0 tx=0x7f0a681160f0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 shutdown_connections 2024-05-10T21:12:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5580a3c561c0 0x5580a40ee2e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580a3cc21b0 0x5580a40ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580a3ca14d0 0x5580a40e30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 --2- 172.21.15.50:0/295325943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580a3c992f0 0x5580a40e2ba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 >> 172.21.15.50:0/295325943 conn(0x5580a3cb6300 msgr2=0x5580a4089370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a50c00640 0 client.26081 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 shutdown_connections 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 1 -- 172.21.15.50:0/295325943 wait complete. 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.668+0000 7f0a74990740 20 client.26081 trim_cache size 0 max 16384 2024-05-10T21:12:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 msgr2=0x7f0a4c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 0x7f0a4c0b4d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5580a406dcd0 tx=0x7f0a58004730 comp rx=0 tx=0).stop 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 msgr2=0x7f0a60140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a68066580 tx=0x7f0a68071820 comp rx=0 tx=0).stop 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 shutdown_connections 2024-05-10T21:12:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a4c0b2870 0x7f0a4c0b4d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a600aad00 0x7f0a601408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a600ad6b0 0x7f0a60147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 --2- 172.21.15.50:0/4050401644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a600a5740 0x7f0a60140390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.670+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 >> 172.21.15.50:0/4050401644 conn(0x7f0a600a09a0 msgr2=0x7f0a600abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.671+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 shutdown_connections 2024-05-10T21:12:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:12.672+0000 7f0a6da00640 1 -- 172.21.15.50:0/4050401644 wait complete. 2024-05-10T21:12:13.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:12 smithi092 ceph-mon[34415]: pgmap v561: 97 pgs: 97 active+clean; 99 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-05-10T21:12:13.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:12 smithi142 ceph-mon[34846]: pgmap v561: 97 pgs: 97 active+clean; 99 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-05-10T21:12:13.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:12 smithi050 ceph-mon[29546]: pgmap v561: 97 pgs: 97 active+clean; 99 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-05-10T21:12:15.226 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:14 smithi050 ceph-mon[29546]: pgmap v562: 97 pgs: 97 active+clean; 110 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-05-10T21:12:15.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:12:15.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:14 smithi092 ceph-mon[34415]: pgmap v562: 97 pgs: 97 active+clean; 110 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-05-10T21:12:15.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:14 smithi142 ceph-mon[34846]: pgmap v562: 97 pgs: 97 active+clean; 110 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-05-10T21:12:17.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:16 smithi092 ceph-mon[34415]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-05-10T21:12:17.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:16 smithi142 ceph-mon[34846]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-05-10T21:12:17.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:16 smithi050 ceph-mon[29546]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-05-10T21:12:18.230 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:19.050 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:18 smithi050 ceph-mon[29546]: pgmap v564: 97 pgs: 97 active+clean; 129 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 371 op/s 2024-05-10T21:12:19.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:18 smithi092 ceph-mon[34415]: pgmap v564: 97 pgs: 97 active+clean; 129 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 371 op/s 2024-05-10T21:12:19.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:18 smithi142 ceph-mon[34846]: pgmap v564: 97 pgs: 97 active+clean; 129 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 371 op/s 2024-05-10T21:12:21.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:20 smithi092 ceph-mon[34415]: pgmap v565: 97 pgs: 97 active+clean; 131 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-10T21:12:21.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:20 smithi142 ceph-mon[34846]: pgmap v565: 97 pgs: 97 active+clean; 131 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-10T21:12:21.381 INFO:tasks.workunit.client.0.smithi050.stdout: 40 380578 331 244182 374 148478 861 2024-05-10T21:12:21.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:20 smithi050 ceph-mon[29546]: pgmap v565: 97 pgs: 97 active+clean; 131 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-10T21:12:23.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:22 smithi092 ceph-mon[34415]: pgmap v566: 97 pgs: 97 active+clean; 132 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-05-10T21:12:23.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:22 smithi142 ceph-mon[34846]: pgmap v566: 97 pgs: 97 active+clean; 132 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-05-10T21:12:23.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:22 smithi050 ceph-mon[29546]: pgmap v566: 97 pgs: 97 active+clean; 132 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-05-10T21:12:25.340 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:24 smithi050 ceph-mon[29546]: pgmap v567: 97 pgs: 97 active+clean; 136 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-05-10T21:12:25.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:24 smithi092 ceph-mon[34415]: pgmap v567: 97 pgs: 97 active+clean; 136 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-05-10T21:12:25.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:24 smithi142 ceph-mon[34846]: pgmap v567: 97 pgs: 97 active+clean; 136 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-05-10T21:12:25.535 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-10T21:12:25.535 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.15 2024-05-10T21:12:25.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.969+0000 7f8f06200640 1 -- 172.21.15.50:0/3840237370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00102340 msgr2=0x7f8f00104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.969+0000 7f8f06200640 1 --2- 172.21.15.50:0/3840237370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00102340 0x7f8f00104760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8eec01a910 tx=0x7f8eec03b470 comp rx=0 tx=0).stop 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.969+0000 7f8f06200640 1 -- 172.21.15.50:0/3840237370 shutdown_connections 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.970+0000 7f8f06200640 1 --2- 172.21.15.50:0/3840237370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f001070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.970+0000 7f8f06200640 1 --2- 172.21.15.50:0/3840237370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00107600 0x7f8f00107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.970+0000 7f8f06200640 1 --2- 172.21.15.50:0/3840237370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00102340 0x7f8f00104760 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.970+0000 7f8f06200640 1 -- 172.21.15.50:0/3840237370 >> 172.21.15.50:0/3840237370 conn(0x7f8f000fdb50 msgr2=0x7f8f00100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.972+0000 7f8f06200640 1 -- 172.21.15.50:0/3840237370 shutdown_connections 2024-05-10T21:12:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.972+0000 7f8f06200640 1 -- 172.21.15.50:0/3840237370 wait complete. 2024-05-10T21:12:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.973+0000 7f8f06200640 1 Processor -- start 2024-05-10T21:12:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.975+0000 7f8f06200640 1 -- start start 2024-05-10T21:12:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.976+0000 7f8f06200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.976+0000 7f8f06200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.976+0000 7f8f06200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 0x7f8f0019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.976+0000 7f8f04e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.976+0000 7f8f04e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42846/0 (socket says 172.21.15.50:42846) 2024-05-10T21:12:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8effe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8effe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39192/0 (socket says 172.21.15.50:39192) 2024-05-10T21:12:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8f04e00640 1 -- 172.21.15.50:0/1916637050 learned_addr learned my addr 172.21.15.50:0/1916637050 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010b0a0 con 0x7f8f00102340 2024-05-10T21:12:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8f05800640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 0x7f8f0019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010af20 con 0x7f8f00107600 2024-05-10T21:12:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.977+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010b220 con 0x7f8f00104ca0 2024-05-10T21:12:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.978+0000 7f8f04e00640 1 -- 172.21.15.50:0/1916637050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 msgr2=0x7f8f00198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.978+0000 7f8f04e00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.978+0000 7f8f04e00640 1 -- 172.21.15.50:0/1916637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 msgr2=0x7f8f0019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.978+0000 7f8f04e00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 0x7f8f0019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.978+0000 7f8f04e00640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0019fbf0 con 0x7f8f00102340 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.979+0000 7f8f04e00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005e60 tx=0x7f8eec004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.979+0000 7f8effe00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.979+0000 7f8f05800640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 0x7f8f0019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.980+0000 7f8efd600640 1 -- 172.21.15.50:0/1916637050 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec04f020 con 0x7f8f00102340 2024-05-10T21:12:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.980+0000 7f8efd600640 1 -- 172.21.15.50:0/1916637050 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec040690 con 0x7f8f00102340 2024-05-10T21:12:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.986+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0019fe80 con 0x7f8f00102340 2024-05-10T21:12:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.986+0000 7f8efd600640 1 -- 172.21.15.50:0/1916637050 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec046070 con 0x7f8f00102340 2024-05-10T21:12:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:26.986+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f001a0360 con 0x7f8f00102340 2024-05-10T21:12:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.007+0000 7f8f0e7ef740 1 Processor -- start 2024-05-10T21:12:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.007+0000 7f8f0e7ef740 1 -- start start 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac08d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d6abfea550 con 0x55d6abcc34d0 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6abf8fbc0 con 0x55d6abcbb2f0 2024-05-10T21:12:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.011+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d6abfe6510 con 0x55d6abce41b0 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.012+0000 7f8f05800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.012+0000 7f8f05800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39198/0 (socket says 172.21.15.50:39198) 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.012+0000 7f8f05800640 1 -- 172.21.15.50:0/1389124945 learned_addr learned my addr 172.21.15.50:0/1389124945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.012+0000 7f8effe00640 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac08d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.012+0000 7f8efd600640 1 -- 172.21.15.50:0/1916637050 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8eec052020 con 0x7f8f00102340 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8f05800640 1 -- 172.21.15.50:0/1389124945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 msgr2=0x55d6ac0868b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8f05800640 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0868b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8f05800640 1 -- 172.21.15.50:0/1389124945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 msgr2=0x55d6ac08d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8f05800640 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac08d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8f05800640 1 -- 172.21.15.50:0/1389124945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6abcfc210 con 0x55d6abce41b0 2024-05-10T21:12:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.013+0000 7f8effe00640 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac08d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:12:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.014+0000 7f8f05800640 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8007f00 tx=0x7f8ef800c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e196b8a95624c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.014+0000 7f8ef2000640 1 -- 172.21.15.50:0/1389124945 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8014070 con 0x55d6abce41b0 2024-05-10T21:12:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.014+0000 7f8ef2000640 1 -- 172.21.15.50:0/1389124945 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef800fd00 con 0x55d6abce41b0 2024-05-10T21:12:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.014+0000 7f8ef2000640 1 -- 172.21.15.50:0/1389124945 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800ec70 con 0x55d6abce41b0 2024-05-10T21:12:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.015+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1389124945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 msgr2=0x55d6ac086330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.015+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8007f00 tx=0x7f8ef800c690 comp rx=0 tx=0).stop 2024-05-10T21:12:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.016+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1389124945 shutdown_connections 2024-05-10T21:12:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.017+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac08d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.017+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.017+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1389124945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac086330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.017+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1389124945 >> 172.21.15.50:0/1389124945 conn(0x55d6abcd8300 msgr2=0x55d6ac081cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.019+0000 7f8efd600640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 0x7f8ee80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.022+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1389124945 shutdown_connections 2024-05-10T21:12:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.022+0000 7f8effe00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 0x7f8ee80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.023+0000 7f8efd600640 1 -- 172.21.15.50:0/1916637050 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8eec113960 con 0x7f8f00102340 2024-05-10T21:12:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.024+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1389124945 wait complete. 2024-05-10T21:12:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.025+0000 7f8f0e7ef740 1 -- start start 2024-05-10T21:12:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.025+0000 7f8effe00640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 0x7f8ee80b4d70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d6ac082a50 tx=0x7f8ef4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.026+0000 7f8f0e7ef740 1 Processor -- start 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac08fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac090420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d6ac08e200 con 0x55d6abcc34d0 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6abfa3660 con 0x55d6abcbb2f0 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d6ac08e380 con 0x55d6abce41b0 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f0e7ef740 10 client.0 _ll_register_callbacks cb 0x55d6abf6d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39204/0 (socket says 172.21.15.50:39204) 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 -- 172.21.15.50:0/2318916399 learned_addr learned my addr 172.21.15.50:0/2318916399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 -- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 msgr2=0x55d6ac08fee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac08fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 -- 172.21.15.50:0/2318916399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 msgr2=0x55d6ac090420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac090420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.027+0000 7f8f05800640 1 -- 172.21.15.50:0/2318916399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6abdc8620 con 0x55d6abce41b0 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.028+0000 7f8f05800640 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800fb80 tx=0x7f8ef8004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd668f457b1e19f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.028+0000 7f8ee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.031+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8014040 con 0x55d6abce41b0 2024-05-10T21:12:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.031+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8ef800edc0 con 0x55d6abce41b0 2024-05-10T21:12:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.031+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef800fd00 con 0x55d6abce41b0 2024-05-10T21:12:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.032+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ef802b450 con 0x55d6abce41b0 2024-05-10T21:12:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.032+0000 7f8f0e7ef740 10 client.26047 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.032+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6ac09a0f0 con 0x55d6abce41b0 2024-05-10T21:12:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.033+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef800fd00 con 0x55d6abce41b0 2024-05-10T21:12:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef8019450 con 0x55d6abce41b0 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 10 client.26047 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 20 client.26047 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abca2d80 0x55d6ac09dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 4 client.26047 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.036+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6abff8160 con 0x55d6abca2d80 2024-05-10T21:12:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.037+0000 7f8f04e00640 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abca2d80 0x55d6ac09dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.038+0000 7f8ee6a00640 10 client.26047 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.038+0000 7f8f04e00640 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abca2d80 0x55d6ac09dcc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f124661754c752c8 server_cookie=2abedb4d05647a67 in_seq=0 out_seq=0 2024-05-10T21:12:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.039+0000 7f8ee6a00640 1 -- 172.21.15.50:0/2318916399 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6abff8160 con 0x55d6abca2d80 2024-05-10T21:12:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.039+0000 7f8ee6a00640 10 client.26047 handle_command_reply: tid=0 2024-05-10T21:12:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.039+0000 7f8f0e7ef740 1 client.26047 shutdown 2024-05-10T21:12:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.039+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 msgr2=0x55d6ac097380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.039+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800fb80 tx=0x7f8ef8004b50 comp rx=0 tx=0).stop 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 shutdown_connections 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abca2d80 0x55d6ac09dcc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8ee6a00640 0 client.26047 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abce41b0 0x55d6ac097380 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcc34d0 0x55d6ac090420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/2318916399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcbb2f0 0x55d6ac08fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 >> 172.21.15.50:0/2318916399 conn(0x55d6abcd8300 msgr2=0x55d6ac080c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 shutdown_connections 2024-05-10T21:12:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.040+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/2318916399 wait complete. 2024-05-10T21:12:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.041+0000 7f8f0e7ef740 20 client.26047 trim_cache size 0 max 16384 2024-05-10T21:12:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.076+0000 7f8f0e7ef740 1 Processor -- start 2024-05-10T21:12:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.076+0000 7f8f0e7ef740 1 -- start start 2024-05-10T21:12:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.077+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac08f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001aba60 con 0x55d6abcd8300 2024-05-10T21:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a9270 con 0x55d6abcc34d0 2024-05-10T21:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f0e7ef740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a7080 con 0x55d6abcbb2f0 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f05800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f05800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54630/0 (socket says 172.21.15.50:54630) 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.078+0000 7f8f05800640 1 -- 172.21.15.50:0/1021475074 learned_addr learned my addr 172.21.15.50:0/1021475074 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8effe00640 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 -- 172.21.15.50:0/1021475074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 msgr2=0x55d6ac0972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 -- 172.21.15.50:0/1021475074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 msgr2=0x55d6ac08f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac08f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 -- 172.21.15.50:0/1021475074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ac082260 con 0x55d6abcc34d0 2024-05-10T21:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8f05800640 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ffd0 tx=0x7f8ef8019bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d890832c8cee81a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8ef1600640 1 -- 172.21.15.50:0/1021475074 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef802b450 con 0x55d6abcc34d0 2024-05-10T21:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8ef1600640 1 -- 172.21.15.50:0/1021475074 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef800fd00 con 0x55d6abcc34d0 2024-05-10T21:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.079+0000 7f8ef1600640 1 -- 172.21.15.50:0/1021475074 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8007750 con 0x55d6abcc34d0 2024-05-10T21:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1021475074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 msgr2=0x55d6ac0977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ffd0 tx=0x7f8ef8019bc0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1021475074 shutdown_connections 2024-05-10T21:12:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac0977e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/1021475074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac08f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.081+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1021475074 >> 172.21.15.50:0/1021475074 conn(0x55d6ac08ab40 msgr2=0x55d6ac08f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.082+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1021475074 shutdown_connections 2024-05-10T21:12:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.082+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/1021475074 wait complete. 2024-05-10T21:12:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.082+0000 7f8f0e7ef740 1 -- start start 2024-05-10T21:12:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.083+0000 7f8f0e7ef740 1 Processor -- start 2024-05-10T21:12:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.084+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.084+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac096c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.084+0000 7f8f0e7ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f05800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f05800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42896/0 (socket says 172.21.15.50:42896) 2024-05-10T21:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f05800640 1 -- 172.21.15.50:0/871002780 learned_addr learned my addr 172.21.15.50:0/871002780 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a7730 con 0x55d6abcd8300 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a8ba0 con 0x55d6abcc34d0 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a7e00 con 0x55d6abcbb2f0 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f0e7ef740 10 client.0 _ll_register_callbacks cb 0x7f8ef400a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.085+0000 7f8f04e00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8effe00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac096c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f05800640 1 -- 172.21.15.50:0/871002780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 msgr2=0x55d6ac0966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f05800640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f05800640 1 -- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 msgr2=0x55d6ac096c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f05800640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac096c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f05800640 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ac0973d0 con 0x55d6abcd8300 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8f04e00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.086+0000 7f8effe00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac096c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.087+0000 7f8f05800640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000f80 tx=0x7f8ef800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e137ad4117c410f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.087+0000 7f8ee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.087+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8017700 con 0x55d6abcd8300 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.087+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8ef800e6b0 con 0x55d6abcd8300 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.088+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef800fd00 con 0x55d6abcd8300 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.088+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ef8019770 con 0x55d6abcd8300 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.088+0000 7f8f0e7ef740 10 client.15951 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.088+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6ac105510 con 0x55d6abcd8300 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef800fd00 con 0x55d6abcd8300 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef8019450 con 0x55d6abcd8300 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 10 client.15951 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 20 client.15951 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abce41b0 0x55d6ac107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 4 client.15951 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:12:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.089+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6abff1fd0 con 0x55d6abce41b0 2024-05-10T21:12:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.090+0000 7f8f04e00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abce41b0 0x55d6ac107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.093+0000 7f8ee6a00640 10 client.15951 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.094+0000 7f8f04e00640 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abce41b0 0x55d6ac107d80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71665ed4aa1fe37d server_cookie=2c641e7490871c02 in_seq=0 out_seq=0 2024-05-10T21:12:27.129 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:26 smithi050 ceph-mon[29546]: pgmap v568: 97 pgs: 97 active+clean; 136 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-05-10T21:12:27.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:26 smithi092 ceph-mon[34415]: pgmap v568: 97 pgs: 97 active+clean; 136 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-05-10T21:12:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:26 smithi142 ceph-mon[34846]: pgmap v568: 97 pgs: 97 active+clean; 136 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-05-10T21:12:27.378 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.429+0000 7f7023c00700 1 -- 172.21.15.50:0/4283561627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c108740 msgr2=0x7f701c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.429+0000 7f7023c00700 1 --2- 172.21.15.50:0/4283561627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c108740 0x7f701c1100a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f701400b0d0 tx=0x7f701400b3e0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.431+0000 7f7023c00700 1 -- 172.21.15.50:0/4283561627 shutdown_connections 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.431+0000 7f7023c00700 1 --2- 172.21.15.50:0/4283561627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c1034c0 0x7f701c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.431+0000 7f7023c00700 1 --2- 172.21.15.50:0/4283561627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c108740 0x7f701c1100a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.431+0000 7f7023c00700 1 --2- 172.21.15.50:0/4283561627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c107df0 0x7f701c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.431+0000 7f7023c00700 1 -- 172.21.15.50:0/4283561627 >> 172.21.15.50:0/4283561627 conn(0x7f701c0fedc0 msgr2=0x7f701c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.433+0000 7f7023c00700 1 -- 172.21.15.50:0/4283561627 shutdown_connections 2024-05-10T21:12:27.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.433+0000 7f7023c00700 1 -- 172.21.15.50:0/4283561627 wait complete. 2024-05-10T21:12:27.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.435+0000 7f7023c00700 1 Processor -- start 2024-05-10T21:12:27.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.435+0000 7f7023c00700 1 -- start start 2024-05-10T21:12:27.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.436+0000 7f7023c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c1034c0 0x7f701c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.437+0000 7f7023c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 0x7f701c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.438+0000 7f7023c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 0x7f701c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.438+0000 7f7023c00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f701c114510 con 0x7f701c1034c0 2024-05-10T21:12:27.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.438+0000 7f7023c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f701c114390 con 0x7f701c108740 2024-05-10T21:12:27.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.438+0000 7f7023c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f701c114690 con 0x7f701c107df0 2024-05-10T21:12:27.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.438+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 0x7f701c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.439+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 0x7f701c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39252/0 (socket says 172.21.15.50:39252) 2024-05-10T21:12:27.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.439+0000 7f701be00700 1 -- 172.21.15.50:0/4246726041 learned_addr learned my addr 172.21.15.50:0/4246726041 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.439+0000 7f7021c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 0x7f701c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 -- 172.21.15.50:0/4246726041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 msgr2=0x7f701c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 0x7f701c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 -- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c1034c0 msgr2=0x7f701c19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c1034c0 0x7f701c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 -- 172.21.15.50:0/4246726041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1a1f70 con 0x7f701c108740 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.440+0000 7f7021c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 0x7f701c1a18d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f701400b820 tx=0x7f70140079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.442+0000 7f7019600700 1 -- 172.21.15.50:0/4246726041 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014021020 con 0x7f701c108740 2024-05-10T21:12:27.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.442+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a2200 con 0x7f701c108740 2024-05-10T21:12:27.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.442+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a9b80 con 0x7f701c108740 2024-05-10T21:12:27.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.442+0000 7f7019600700 1 -- 172.21.15.50:0/4246726041 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014004c50 con 0x7f701c108740 2024-05-10T21:12:27.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.442+0000 7f7019600700 1 -- 172.21.15.50:0/4246726041 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014021020 con 0x7f701c108740 2024-05-10T21:12:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.453+0000 7f7019600700 1 -- 172.21.15.50:0/4246726041 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7014024020 con 0x7f701c108740 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.460+0000 7f703270cb80 1 Processor -- start 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 -- start start 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda8c460 0x55bedda8c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bedd9a5a20 con 0x55bedda8c460 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.461+0000 7f703270cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bedd943760 con 0x55bedda92400 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.462+0000 7f703270cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bedd9a19e0 con 0x55bedd905e40 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.462+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.462+0000 7f7021200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.462+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39260/0 (socket says 172.21.15.50:39260) 2024-05-10T21:12:27.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.462+0000 7f701be00700 1 -- 172.21.15.50:0/1876676818 learned_addr learned my addr 172.21.15.50:0/1876676818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f7019600700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 0x7f70040b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda8c460 0x55bedda8c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 -- 172.21.15.50:0/1876676818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 msgr2=0x55bedda8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 -- 172.21.15.50:0/1876676818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda8c460 msgr2=0x55bedda8c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda8c460 0x55bedda8c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 -- 172.21.15.50:0/1876676818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedd8f28d0 con 0x55bedd905e40 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f7019600700 1 -- 172.21.15.50:0/4246726041 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f701400f020 con 0x7f701c108740 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f701be00700 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7010012570 tx=0x7f7010005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4fbbcc06287c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.467+0000 7f7021c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 0x7f70040b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.469+0000 7f700e000700 1 -- 172.21.15.50:0/1876676818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010004830 con 0x55bedd905e40 2024-05-10T21:12:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.469+0000 7f7021c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 0x7f70040b5010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7014007fd0 tx=0x7f70140099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.469+0000 7f700e000700 1 -- 172.21.15.50:0/1876676818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7010005020 con 0x55bedd905e40 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.469+0000 7f700e000700 1 -- 172.21.15.50:0/1876676818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701000f650 con 0x55bedd905e40 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.470+0000 7f703270cb80 1 -- 172.21.15.50:0/1876676818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 msgr2=0x55bedda91ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.470+0000 7f703270cb80 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7010012570 tx=0x7f7010005d00 comp rx=0 tx=0).stop 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.471+0000 7f703270cb80 1 -- 172.21.15.50:0/1876676818 shutdown_connections 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.471+0000 7f703270cb80 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda8c460 0x55bedda8c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.471+0000 7f703270cb80 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda91ec0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.471+0000 7f703270cb80 1 --2- 172.21.15.50:0/1876676818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.471+0000 7f703270cb80 1 -- 172.21.15.50:0/1876676818 >> 172.21.15.50:0/1876676818 conn(0x55bedd944a80 msgr2=0x55bedda7e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.472+0000 7f703270cb80 1 -- 172.21.15.50:0/1876676818 shutdown_connections 2024-05-10T21:12:27.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.473+0000 7f703270cb80 1 -- 172.21.15.50:0/1876676818 wait complete. 2024-05-10T21:12:27.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.474+0000 7f703270cb80 1 -- start start 2024-05-10T21:12:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.475+0000 7f703270cb80 1 Processor -- start 2024-05-10T21:12:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.476+0000 7f703270cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.476+0000 7f703270cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.477+0000 7f703270cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda92400 0x55bedda8aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f703270cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bedda839b0 con 0x55bedda92400 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f7021200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f701be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39274/0 (socket says 172.21.15.50:39274) 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f7021200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54678/0 (socket says 172.21.15.50:54678) 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.478+0000 7f701be00700 1 -- 172.21.15.50:0/1928377065 learned_addr learned my addr 172.21.15.50:0/1928377065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.481+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bedd90bb20 con 0x55bedda8c460 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.481+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bedda83b30 con 0x55bedd905e40 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.482+0000 7f703270cb80 10 client.0 _ll_register_callbacks cb 0x55bedd939380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.482+0000 7f701be00700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda92400 0x55bedda8aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f701be00700 1 -- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 msgr2=0x55bedda967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f701be00700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f701be00700 1 -- 172.21.15.50:0/1928377065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda92400 msgr2=0x55bedda8aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f701be00700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda92400 0x55bedda8aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f701be00700 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedd96b420 con 0x55bedd905e40 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.483+0000 7f7021200700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.484+0000 7f701be00700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7010005fa0 tx=0x7f7010014e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44af9fd9b9941d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.484+0000 7f7002a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.484+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701002c060 con 0x55bedd905e40 2024-05-10T21:12:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.484+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7010004a50 con 0x55bedd905e40 2024-05-10T21:12:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.485+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7010005020 con 0x55bedd905e40 2024-05-10T21:12:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.485+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7010022070 con 0x55bedd905e40 2024-05-10T21:12:27.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.485+0000 7f703270cb80 10 client.26062 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:27.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.486+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55beddaac2c0 con 0x55bedd905e40 2024-05-10T21:12:27.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.488+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7010005020 con 0x55bedd905e40 2024-05-10T21:12:27.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.488+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f701001c020 con 0x55bedd905e40 2024-05-10T21:12:27.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 10 client.26062 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:27.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:27.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 20 client.26062 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:27.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddaae840 0x55beddab0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 4 client.26062 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:27.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.491+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bedda50070 con 0x55beddaae840 2024-05-10T21:12:27.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.493+0000 7f7021200700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddaae840 0x55beddab0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.494+0000 7f7021200700 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddaae840 0x55beddab0c60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b1d27eba209c40 server_cookie=ec632ddb152616ea in_seq=0 out_seq=0 2024-05-10T21:12:27.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.494+0000 7f7002a00700 10 client.26062 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.593+0000 7f8ee6a00640 1 -- 172.21.15.50:0/871002780 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6abff1fd0 con 0x55d6abce41b0 2024-05-10T21:12:27.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.593+0000 7f7002a00700 1 -- 172.21.15.50:0/1928377065 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bedda50070 con 0x55beddaae840 2024-05-10T21:12:27.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.593+0000 7f7002a00700 10 client.26062 handle_command_reply: tid=0 2024-05-10T21:12:27.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.593+0000 7f703270cb80 1 client.26062 shutdown 2024-05-10T21:12:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.593+0000 7f8ee6a00640 10 client.15951 handle_command_reply: tid=0 2024-05-10T21:12:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f8f0e7ef740 1 client.15951 shutdown 2024-05-10T21:12:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.596+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 msgr2=0x55d6ac104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.596+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000f80 tx=0x7f8ef800db00 comp rx=0 tx=0).stop 2024-05-10T21:12:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 msgr2=0x55bedda96260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7010005fa0 tx=0x7f7010014e80 comp rx=0 tx=0).stop 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 shutdown_connections 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddaae840 0x55beddab0c60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedda92400 0x55bedda8aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedd905e40 0x55bedda96260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 --2- 172.21.15.50:0/1928377065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 >> 172.21.15.50:0/1928377065 conn(0x55bedd944a80 msgr2=0x55bedda78430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f7002a00700 0 client.26062 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 shutdown_connections 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 1 -- 172.21.15.50:0/1928377065 wait complete. 2024-05-10T21:12:27.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.594+0000 7f703270cb80 20 client.26062 trim_cache size 0 max 16384 2024-05-10T21:12:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.601+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 shutdown_connections 2024-05-10T21:12:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.601+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55d6abce41b0 0x55d6ac107d80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.603+0000 7f8ee6a00640 0 client.15951 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.603+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d6abcd8300 0x55d6ac104020 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.603+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6abcc34d0 0x55d6ac096c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.603+0000 7f8f0e7ef740 1 --2- 172.21.15.50:0/871002780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6abcbb2f0 0x55d6ac0966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.603+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 >> 172.21.15.50:0/871002780 conn(0x55d6ac08ab40 msgr2=0x55d6ac084ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.622+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 shutdown_connections 2024-05-10T21:12:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.623+0000 7f8f0e7ef740 1 -- 172.21.15.50:0/871002780 wait complete. 2024-05-10T21:12:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.623+0000 7f8f0e7ef740 20 client.15951 trim_cache size 0 max 16384 2024-05-10T21:12:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:12:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:12:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:12:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.626+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 msgr2=0x7f8ee80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.626+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 0x7f8ee80b4d70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d6ac082a50 tx=0x7f8ef4008040 comp rx=0 tx=0).stop 2024-05-10T21:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.626+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 msgr2=0x7f8f001980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.626+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005e60 tx=0x7f8eec004c60 comp rx=0 tx=0).stop 2024-05-10T21:12:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 shutdown_connections 2024-05-10T21:12:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8ee80b2860 0x7f8ee80b4d70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f00104ca0 0x7f8f00198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f00107600 0x7f8f0019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 --2- 172.21.15.50:0/1916637050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f00102340 0x7f8f001980e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.631+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 >> 172.21.15.50:0/1916637050 conn(0x7f8f000fdb50 msgr2=0x7f8f000fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.634+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 shutdown_connections 2024-05-10T21:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:27.634+0000 7f8f06200640 1 -- 172.21.15.50:0/1916637050 wait complete. 2024-05-10T21:12:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.647+0000 7f703270cb80 1 Processor -- start 2024-05-10T21:12:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 -- start start 2024-05-10T21:12:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda93b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda92400 0x55beddab62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00fc70 con 0x55bedd905e40 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00d440 con 0x55bedda8c460 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.649+0000 7f703270cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00b290 con 0x55bedda92400 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.650+0000 7f7021200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.650+0000 7f7021200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54682/0 (socket says 172.21.15.50:54682) 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.650+0000 7f7021200700 1 -- 172.21.15.50:0/2627661599 learned_addr learned my addr 172.21.15.50:0/2627661599 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.650+0000 7f701be00700 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda93b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f7021200700 1 -- 172.21.15.50:0/2627661599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda92400 msgr2=0x55beddab62c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:12:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f7021200700 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda92400 0x55beddab62c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f7021200700 1 -- 172.21.15.50:0/2627661599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 msgr2=0x55bedda93b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f7021200700 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda93b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f7021200700 1 -- 172.21.15.50:0/2627661599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedda96f80 con 0x55bedda8c460 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.654+0000 7f701be00700 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda93b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.655+0000 7f7021200700 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7008005d90 tx=0x7f700800b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d2979194708fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.655+0000 7f700d600700 1 -- 172.21.15.50:0/2627661599 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008013c60 con 0x55bedda8c460 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.656+0000 7f700d600700 1 -- 172.21.15.50:0/2627661599 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7008013e00 con 0x55bedda8c460 2024-05-10T21:12:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.656+0000 7f700d600700 1 -- 172.21.15.50:0/2627661599 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700801a330 con 0x55bedda8c460 2024-05-10T21:12:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.658+0000 7f703270cb80 1 -- 172.21.15.50:0/2627661599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 msgr2=0x55bedda78b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.658+0000 7f703270cb80 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7008005d90 tx=0x7f700800b4c0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.659+0000 7f703270cb80 1 -- 172.21.15.50:0/2627661599 shutdown_connections 2024-05-10T21:12:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.659+0000 7f703270cb80 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda92400 0x55beddab62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.659+0000 7f703270cb80 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda93b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.659+0000 7f703270cb80 1 --2- 172.21.15.50:0/2627661599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda8c460 0x55bedda78b40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.659+0000 7f703270cb80 1 -- 172.21.15.50:0/2627661599 >> 172.21.15.50:0/2627661599 conn(0x55bedd944a80 msgr2=0x55bedda78430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.662+0000 7f703270cb80 1 -- 172.21.15.50:0/2627661599 shutdown_connections 2024-05-10T21:12:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.662+0000 7f703270cb80 1 -- 172.21.15.50:0/2627661599 wait complete. 2024-05-10T21:12:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.663+0000 7f703270cb80 1 -- start start 2024-05-10T21:12:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.663+0000 7f703270cb80 1 Processor -- start 2024-05-10T21:12:27.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda84c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00b900 con 0x55bedd905e40 2024-05-10T21:12:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00cd70 con 0x55bedda92400 2024-05-10T21:12:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00bfd0 con 0x55bedda8c460 2024-05-10T21:12:27.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.665+0000 7f703270cb80 10 client.0 _ll_register_callbacks cb 0x55bedd939380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.668+0000 7f7021200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.668+0000 7f7021200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39290/0 (socket says 172.21.15.50:39290) 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.668+0000 7f7021200700 1 -- 172.21.15.50:0/3335979898 learned_addr learned my addr 172.21.15.50:0/3335979898 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.668+0000 7f701be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.670+0000 7f701be00700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda84c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.670+0000 7f7021200700 1 -- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 msgr2=0x55bedda84c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.670+0000 7f7021200700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda84c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.671+0000 7f7021200700 1 -- 172.21.15.50:0/3335979898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 msgr2=0x55bedda8b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.671+0000 7f7021200700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda8b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.671+0000 7f7021200700 1 -- 172.21.15.50:0/3335979898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedda853d0 con 0x55bedda8c460 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.672+0000 7f701be00700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda84c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.677+0000 7f7002a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.682+0000 7f7021200700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f700800be30 tx=0x7f700800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b0ab7820d148d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.682+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008005420 con 0x55bedda8c460 2024-05-10T21:12:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.682+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008013e00 con 0x55bedda8c460 2024-05-10T21:12:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.682+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7008005020 con 0x55bedda8c460 2024-05-10T21:12:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.683+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7008005420 con 0x55bedda8c460 2024-05-10T21:12:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.683+0000 7f703270cb80 10 client.26065 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.683+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55beddb18e70 con 0x55bedda8c460 2024-05-10T21:12:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7008013e00 con 0x55bedda8c460 2024-05-10T21:12:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f700802a020 con 0x55bedda8c460 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 10 client.26065 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 20 client.26065 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddb19450 0x55beddb1b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 4 client.26065 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:12:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.685+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bedd9b3400 con 0x55beddb19450 2024-05-10T21:12:27.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.689+0000 7f701be00700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddb19450 0x55beddb1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.690+0000 7f701be00700 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddb19450 0x55beddb1b870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb54f4c41fd21ba6 server_cookie=5cde0d84791c1ba8 in_seq=0 out_seq=0 2024-05-10T21:12:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.690+0000 7f7002a00700 10 client.26065 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.690+0000 7f7002a00700 1 -- 172.21.15.50:0/3335979898 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bedd9b3400 con 0x55beddb19450 2024-05-10T21:12:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.690+0000 7f7002a00700 10 client.26065 handle_command_reply: tid=0 2024-05-10T21:12:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.690+0000 7f703270cb80 1 client.26065 shutdown 2024-05-10T21:12:27.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 msgr2=0x55bedda846f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f700800be30 tx=0x7f700800bf10 comp rx=0 tx=0).stop 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 shutdown_connections 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55beddb19450 0x55beddb1b870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bedda92400 0x55bedda84c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bedd905e40 0x55bedda8b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 --2- 172.21.15.50:0/3335979898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bedda8c460 0x55bedda846f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.692+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 >> 172.21.15.50:0/3335979898 conn(0x55bedd944a80 msgr2=0x55bedda7de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.693+0000 7f7002a00700 0 client.26065 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.694+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 shutdown_connections 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.694+0000 7f703270cb80 1 -- 172.21.15.50:0/3335979898 wait complete. 2024-05-10T21:12:27.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.694+0000 7f703270cb80 20 client.26065 trim_cache size 0 max 16384 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 msgr2=0x7f70040b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 0x7f70040b5010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7014007fd0 tx=0x7f70140099e0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 msgr2=0x7f701c1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 0x7f701c1a18d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f701400b820 tx=0x7f70140079a0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 shutdown_connections 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f70040b2b00 0x7f70040b5010 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f701c108740 0x7f701c1a18d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c1034c0 0x7f701c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 --2- 172.21.15.50:0/4246726041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f701c107df0 0x7f701c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.696+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 >> 172.21.15.50:0/4246726041 conn(0x7f701c0fedc0 msgr2=0x7f701c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.697+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 shutdown_connections 2024-05-10T21:12:27.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:27.697+0000 7f7023c00700 1 -- 172.21.15.50:0/4246726041 wait complete. 2024-05-10T21:12:29.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:28 smithi092 ceph-mon[34415]: pgmap v569: 97 pgs: 97 active+clean; 137 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-10T21:12:29.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:28 smithi142 ceph-mon[34846]: pgmap v569: 97 pgs: 97 active+clean; 137 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-10T21:12:29.423 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:28 smithi050 ceph-mon[29546]: pgmap v569: 97 pgs: 97 active+clean; 137 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-10T21:12:29.644 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:12:29.644 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:12:30.783 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.872+0000 7fe861e00700 1 -- 172.21.15.50:0/3179072999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c10e900 msgr2=0x7fe85c110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.872+0000 7fe861e00700 1 --2- 172.21.15.50:0/3179072999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c10e900 0x7fe85c110dc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe85400b0a0 tx=0x7fe85400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 -- 172.21.15.50:0/3179072999 shutdown_connections 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 --2- 172.21.15.50:0/3179072999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c10e900 0x7fe85c110dc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 --2- 172.21.15.50:0/3179072999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c105520 0x7fe85c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 --2- 172.21.15.50:0/3179072999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105e70 0x7fe85c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 -- 172.21.15.50:0/3179072999 >> 172.21.15.50:0/3179072999 conn(0x7fe85c0fedc0 msgr2=0x7fe85c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:30.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 -- 172.21.15.50:0/3179072999 shutdown_connections 2024-05-10T21:12:30.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.873+0000 7fe861e00700 1 -- 172.21.15.50:0/3179072999 wait complete. 2024-05-10T21:12:30.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.874+0000 7fe861e00700 1 Processor -- start 2024-05-10T21:12:30.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.874+0000 7fe861e00700 1 -- start start 2024-05-10T21:12:30.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105520 0x7fe85c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:30.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:30.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c10e900 0x7fe85c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c114440 con 0x7fe85c105520 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1142c0 con 0x7fe85c105e70 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe861e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1145c0 con 0x7fe85c10e900 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe85b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c10e900 0x7fe85c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.875+0000 7fe85a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:54696/0 (socket says 172.21.15.50:54696) 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 -- 172.21.15.50:0/766320332 learned_addr learned my addr 172.21.15.50:0/766320332 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85aa00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105520 0x7fe85c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:30.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 -- 172.21.15.50:0/766320332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c10e900 msgr2=0x7fe85c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c10e900 0x7fe85c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 -- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105520 msgr2=0x7fe85c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105520 0x7fe85c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 -- 172.21.15.50:0/766320332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c19fc60 con 0x7fe85c105e70 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe85a000700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe84800e9a0 tx=0x7fe84800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:30.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84800cbd0 con 0x7fe85c105e70 2024-05-10T21:12:30.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84800cd70 con 0x7fe85c105e70 2024-05-10T21:12:30.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848018680 con 0x7fe85c105e70 2024-05-10T21:12:30.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.876+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c19ff50 con 0x7fe85c105e70 2024-05-10T21:12:30.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.877+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c103140 con 0x7fe85c105e70 2024-05-10T21:12:30.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.878+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85c06be30 con 0x7fe85c105e70 2024-05-10T21:12:30.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.881+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe848010430 con 0x7fe85c105e70 2024-05-10T21:12:30.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.883+0000 7fe860a00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 0x7fe8400b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:30.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.883+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe8480e2900 con 0x7fe85c105e70 2024-05-10T21:12:30.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.884+0000 7fe85aa00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 0x7fe8400b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:30.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.884+0000 7fe85aa00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 0x7fe8400b91a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005950 tx=0x7fe84c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:30.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:30.885+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8480aa5f0 con 0x7fe85c105e70 2024-05-10T21:12:31.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:30 smithi050 ceph-mon[29546]: pgmap v570: 97 pgs: 97 active+clean; 139 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-10T21:12:31.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:12:31.343 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:11:36.879959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:12:31.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.340+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe85c0512c0 con 0x7fe85c105e70 2024-05-10T21:12:31.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.341+0000 7fe860a00700 1 -- 172.21.15.50:0/766320332 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11858 (secure 0 0 0) 0x7fe8480a9ef0 con 0x7fe85c105e70 2024-05-10T21:12:31.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.347+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 msgr2=0x7fe8400b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:31.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.347+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 0x7fe8400b91a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005950 tx=0x7fe84c0058e0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.347+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 msgr2=0x7fe85c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:31.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.347+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe84800e9a0 tx=0x7fe84800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 shutdown_connections 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe8400b6c90 0x7fe8400b91a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe85c10e900 0x7fe85c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe85c105520 0x7fe85c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 --2- 172.21.15.50:0/766320332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe85c105e70 0x7fe85c19b330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:31.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.348+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 >> 172.21.15.50:0/766320332 conn(0x7fe85c0fedc0 msgr2=0x7fe85c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:31.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.366+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 shutdown_connections 2024-05-10T21:12:31.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:31.366+0000 7fe861e00700 1 -- 172.21.15.50:0/766320332 wait complete. 2024-05-10T21:12:31.366 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 31 2024-05-10T21:12:31.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:30 smithi092 ceph-mon[34415]: pgmap v570: 97 pgs: 97 active+clean; 139 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-10T21:12:31.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:30 smithi142 ceph-mon[34846]: pgmap v570: 97 pgs: 97 active+clean; 139 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-10T21:12:31.388 INFO:tasks.workunit.client.0.smithi050.stdout: 44 434982 298 277648 279 169391 323 2024-05-10T21:12:32.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:32 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/766320332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:32.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:32 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/766320332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:32.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:32 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/766320332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:33.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:33 smithi092 ceph-mon[34415]: pgmap v571: 97 pgs: 97 active+clean; 149 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-05-10T21:12:33.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:33 smithi142 ceph-mon[34846]: pgmap v571: 97 pgs: 97 active+clean; 149 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-05-10T21:12:33.420 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:33 smithi050 ceph-mon[29546]: pgmap v571: 97 pgs: 97 active+clean; 149 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-05-10T21:12:34.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:34 smithi050 ceph-mon[29546]: pgmap v572: 97 pgs: 97 active+clean; 163 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 380 op/s 2024-05-10T21:12:34.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:34 smithi092 ceph-mon[34415]: pgmap v572: 97 pgs: 97 active+clean; 163 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 380 op/s 2024-05-10T21:12:34.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:34 smithi142 ceph-mon[34846]: pgmap v572: 97 pgs: 97 active+clean; 163 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 380 op/s 2024-05-10T21:12:34.675 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0' 2024-05-10T21:12:34.676 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.404+0000 7fbc05e00700 1 -- 172.21.15.50:0/1764042830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf809c610 msgr2=0x7fbbf809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.404+0000 7fbc05e00700 1 --2- 172.21.15.50:0/1764042830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf809c610 0x7fbbf809ca20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 -- 172.21.15.50:0/1764042830 shutdown_connections 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 --2- 172.21.15.50:0/1764042830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf809c610 0x7fbbf809ca20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 --2- 172.21.15.50:0/1764042830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf8096f40 0x7fbbf80a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 --2- 172.21.15.50:0/1764042830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf80965f0 0x7fbbf8096a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 -- 172.21.15.50:0/1764042830 >> 172.21.15.50:0/1764042830 conn(0x7fbbf800c300 msgr2=0x7fbbf800c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 -- 172.21.15.50:0/1764042830 shutdown_connections 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.405+0000 7fbc05e00700 1 -- 172.21.15.50:0/1764042830 wait complete. 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.407+0000 7fbc05e00700 1 Processor -- start 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.407+0000 7fbc05e00700 1 -- start start 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf8096f40 0x7fbbf812f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf809c610 0x7fbbf8133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf80a6850 con 0x7fbbf8096f40 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf80a66d0 con 0x7fbbf80965f0 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc05e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf80a69d0 con 0x7fbbf809c610 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.408+0000 7fbc04a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59258/0 (socket says 172.21.15.50:59258) 2024-05-10T21:12:36.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 -- 172.21.15.50:0/750574994 learned_addr learned my addr 172.21.15.50:0/750574994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 -- 172.21.15.50:0/750574994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf809c610 msgr2=0x7fbbf8133c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf809c610 0x7fbbf8133c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 -- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf8096f40 msgr2=0x7fbbf812f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf8096f40 0x7fbbf812f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 -- 172.21.15.50:0/750574994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf81342f0 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.409+0000 7fbc04a00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005b40 tx=0x7fbbf4005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.410+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4027070 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.410+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf4004030 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.410+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4022d80 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.410+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf8134580 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.411+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf8134a90 con 0x7fbbf80965f0 2024-05-10T21:12:36.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.413+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf8004910 con 0x7fbbf80965f0 2024-05-10T21:12:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.418+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbbf4022470 con 0x7fbbf80965f0 2024-05-10T21:12:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.421+0000 7fbbfd600700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 0x7fbbe80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.422+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fbbf40e3a80 con 0x7fbbf80965f0 2024-05-10T21:12:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.422+0000 7fbbffe00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 0x7fbbe80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.423+0000 7fbbffe00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 0x7fbbe80b4e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c970 tx=0x7fbbec006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:36.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:36.424+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbf4013070 con 0x7fbbf80965f0 2024-05-10T21:12:37.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.002+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbf813bfc0 con 0x7fbbf80965f0 2024-05-10T21:12:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:12:37.008 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:11:36.879959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.003+0000 7fbbfd600700 1 -- 172.21.15.50:0/750574994 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11858 (secure 0 0 0) 0x7fbbf40ab800 con 0x7fbbf80965f0 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 msgr2=0x7fbbe80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 0x7fbbe80b4e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c970 tx=0x7fbbec006cd0 comp rx=0 tx=0).stop 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 msgr2=0x7fbbf812f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005b40 tx=0x7fbbf4005640 comp rx=0 tx=0).stop 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 shutdown_connections 2024-05-10T21:12:37.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbe80b2980 0x7fbbe80b4e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf80965f0 0x7fbbf812f420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf809c610 0x7fbbf8133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 --2- 172.21.15.50:0/750574994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf8096f40 0x7fbbf812f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.008+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 >> 172.21.15.50:0/750574994 conn(0x7fbbf800c300 msgr2=0x7fbbf80a1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.010+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 shutdown_connections 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:37.010+0000 7fbc05e00700 1 -- 172.21.15.50:0/750574994 wait complete. 2024-05-10T21:12:37.010 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 31 2024-05-10T21:12:37.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:36 smithi092 ceph-mon[34415]: pgmap v573: 97 pgs: 97 active+clean; 174 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 436 op/s 2024-05-10T21:12:37.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:36 smithi142 ceph-mon[34846]: pgmap v573: 97 pgs: 97 active+clean; 174 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 436 op/s 2024-05-10T21:12:37.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:36 smithi050 ceph-mon[29546]: pgmap v573: 97 pgs: 97 active+clean; 174 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 436 op/s 2024-05-10T21:12:37.854 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:37 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/750574994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:38.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:37 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/750574994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:38.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:37 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/750574994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:12:39.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:38 smithi092 ceph-mon[34415]: pgmap v574: 97 pgs: 97 active+clean; 181 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-05-10T21:12:39.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:38 smithi142 ceph-mon[34846]: pgmap v574: 97 pgs: 97 active+clean; 181 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-05-10T21:12:39.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:38 smithi050 ceph-mon[29546]: pgmap v574: 97 pgs: 97 active+clean; 181 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-05-10T21:12:39.692 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:39.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:12:41.069 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:40 smithi050 ceph-mon[29546]: pgmap v575: 97 pgs: 97 active+clean; 188 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-05-10T21:12:41.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:40 smithi092 ceph-mon[34415]: pgmap v575: 97 pgs: 97 active+clean; 188 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-05-10T21:12:41.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:40 smithi142 ceph-mon[34846]: pgmap v575: 97 pgs: 97 active+clean; 188 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-05-10T21:12:41.381 INFO:tasks.workunit.client.0.smithi050.stdout: 52 511704 412 324295 378 191601 360 2024-05-10T21:12:42.721 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 -- 172.21.15.50:0/97844575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410a470 msgr2=0x7f2d2410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 --2- 172.21.15.50:0/97844575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410a470 0x7f2d2410ad80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009900 tx=0x7f2d1002a420 comp rx=0 tx=0).stop 2024-05-10T21:12:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 -- 172.21.15.50:0/97844575 shutdown_connections 2024-05-10T21:12:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 --2- 172.21.15.50:0/97844575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 --2- 172.21.15.50:0/97844575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410b2c0 0x7f2d2410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 --2- 172.21.15.50:0/97844575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410a470 0x7f2d2410ad80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.807+0000 7f2d28a00640 1 -- 172.21.15.50:0/97844575 >> 172.21.15.50:0/97844575 conn(0x7f2d24070460 msgr2=0x7f2d24072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.814+0000 7f2d28a00640 1 -- 172.21.15.50:0/97844575 shutdown_connections 2024-05-10T21:12:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.814+0000 7f2d28a00640 1 -- 172.21.15.50:0/97844575 wait complete. 2024-05-10T21:12:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.816+0000 7f2d28a00640 1 Processor -- start 2024-05-10T21:12:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.816+0000 7f2d28a00640 1 -- start start 2024-05-10T21:12:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.819+0000 7f2d28a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.819+0000 7f2d28a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410b2c0 0x7f2d241a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d22800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d22800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44588/0 (socket says 172.21.15.50:44588) 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d22800640 1 -- 172.21.15.50:0/229137479 learned_addr learned my addr 172.21.15.50:0/229137479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d21e00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410b2c0 0x7f2d241a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2411e3c0 con 0x7f2d2410a470 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2411e240 con 0x7f2d2410b2c0 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2411e540 con 0x7f2d2410ffe0 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.820+0000 7f2d23200640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 -- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 msgr2=0x7f2d241b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 -- 172.21.15.50:0/229137479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410b2c0 msgr2=0x7f2d241a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410b2c0 0x7f2d241a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241b07d0 con 0x7f2d2410a470 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.821+0000 7f2d22800640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009850 tx=0x7f2d10003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.822+0000 7f2d23200640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.827+0000 7f2d1b400640 1 -- 172.21.15.50:0/229137479 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1003f030 con 0x7f2d2410a470 2024-05-10T21:12:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.827+0000 7f2d1b400640 1 -- 172.21.15.50:0/229137479 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d10032cf0 con 0x7f2d2410a470 2024-05-10T21:12:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.827+0000 7f2d1b400640 1 -- 172.21.15.50:0/229137479 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10035070 con 0x7f2d2410a470 2024-05-10T21:12:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.830+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241b0a60 con 0x7f2d2410a470 2024-05-10T21:12:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.832+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241b0e10 con 0x7f2d2410a470 2024-05-10T21:12:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.847+0000 7f2d1b400640 1 -- 172.21.15.50:0/229137479 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d10042030 con 0x7f2d2410a470 2024-05-10T21:12:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.855+0000 7f2d30433740 1 Processor -- start 2024-05-10T21:12:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.855+0000 7f2d1b400640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 0x7f2d080b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.855+0000 7f2d30433740 1 -- start start 2024-05-10T21:12:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.858+0000 7f2d30433740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e21e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.858+0000 7f2d30433740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e182f0 0x55a5e21e3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d30433740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d30433740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a5e2147550 con 0x55a5e1e204d0 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d30433740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a5e20ecbc0 con 0x55a5e1e411b0 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d30433740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a5e2143510 con 0x55a5e1e182f0 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d22800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e21e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d22800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e21e32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59292/0 (socket says 172.21.15.50:59292) 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d22800640 1 -- 172.21.15.50:0/3768504985 learned_addr learned my addr 172.21.15.50:0/3768504985 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.859+0000 7f2d23200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d23200640 1 -- 172.21.15.50:0/3768504985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e182f0 msgr2=0x55a5e21e3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d23200640 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e182f0 0x55a5e21e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d23200640 1 -- 172.21.15.50:0/3768504985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 msgr2=0x55a5e21e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d23200640 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e21e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.860+0000 7f2d23200640 1 -- 172.21.15.50:0/3768504985 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e1e59210 con 0x55a5e1e204d0 2024-05-10T21:12:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.861+0000 7f2d23200640 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21ea770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400ce20 tx=0x7f2d1400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bbbd4ad22590b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.862+0000 7f2d0f400640 1 -- 172.21.15.50:0/3768504985 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005220 con 0x55a5e1e204d0 2024-05-10T21:12:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.862+0000 7f2d0f400640 1 -- 172.21.15.50:0/3768504985 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004c50 con 0x55a5e1e204d0 2024-05-10T21:12:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.862+0000 7f2d0f400640 1 -- 172.21.15.50:0/3768504985 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140107f0 con 0x55a5e1e204d0 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.863+0000 7f2d21e00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 0x7f2d080b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.864+0000 7f2d1b400640 1 -- 172.21.15.50:0/229137479 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2d10102240 con 0x7f2d2410a470 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.865+0000 7f2d30433740 1 -- 172.21.15.50:0/3768504985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 msgr2=0x55a5e21ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.865+0000 7f2d30433740 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21ea770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400ce20 tx=0x7f2d1400e720 comp rx=0 tx=0).stop 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.866+0000 7f2d30433740 1 -- 172.21.15.50:0/3768504985 shutdown_connections 2024-05-10T21:12:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.866+0000 7f2d30433740 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e182f0 0x55a5e21e3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.866+0000 7f2d30433740 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21ea770 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.866+0000 7f2d30433740 1 --2- 172.21.15.50:0/3768504985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e21e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.866+0000 7f2d30433740 1 -- 172.21.15.50:0/3768504985 >> 172.21.15.50:0/3768504985 conn(0x55a5e1e35300 msgr2=0x55a5e21dec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.868+0000 7f2d21e00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 0x7f2d080b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007350 tx=0x7f2d1c0072a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.870+0000 7f2d30433740 1 -- 172.21.15.50:0/3768504985 shutdown_connections 2024-05-10T21:12:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.874+0000 7f2d30433740 1 -- 172.21.15.50:0/3768504985 wait complete. 2024-05-10T21:12:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.875+0000 7f2d30433740 1 -- start start 2024-05-10T21:12:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.878+0000 7f2d30433740 1 Processor -- start 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e204d0 0x55a5e21ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e411b0 0x55a5e21ed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a5e21eb1c0 con 0x55a5e1e411b0 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a5e2100660 con 0x55a5e1dffd80 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a5e21eb340 con 0x55a5e1e204d0 2024-05-10T21:12:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.882+0000 7f2d30433740 10 client.0 _ll_register_callbacks cb 0x55a5e20ca400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.883+0000 7f2d23200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.883+0000 7f2d23200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59306/0 (socket says 172.21.15.50:59306) 2024-05-10T21:12:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.883+0000 7f2d23200640 1 -- 172.21.15.50:0/633139821 learned_addr learned my addr 172.21.15.50:0/633139821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.884+0000 7f2d21e00640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e411b0 0x55a5e21ed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d22800640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e204d0 0x55a5e21ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d23200640 1 -- 172.21.15.50:0/633139821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e204d0 msgr2=0x55a5e21ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d23200640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e204d0 0x55a5e21ecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d23200640 1 -- 172.21.15.50:0/633139821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e411b0 msgr2=0x55a5e21ed4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d23200640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e411b0 0x55a5e21ed4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.885+0000 7f2d23200640 1 -- 172.21.15.50:0/633139821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e1f25620 con 0x55a5e1dffd80 2024-05-10T21:12:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.890+0000 7f2d23200640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005280 tx=0x7f2d140052b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b2f2e8a8074b112 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.893+0000 7f2d03e00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.893+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400c7b0 con 0x55a5e1dffd80 2024-05-10T21:12:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.893+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2d14004e00 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.898+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14004c50 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.898+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d14010530 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.901+0000 7f2d30433740 10 client.26126 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.901+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5e21f7330 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.901+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004c50 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.901+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d1402f700 con 0x55a5e1dffd80 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 10 client.26126 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 20 client.26126 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e1dd51c0 0x55a5e21fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 4 client.26126 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e2155160 con 0x55a5e1dd51c0 2024-05-10T21:12:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.903+0000 7f2d22800640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e1dd51c0 0x55a5e21fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.904+0000 7f2d22800640 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e1dd51c0 0x55a5e21fa880 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b08985836ae358 server_cookie=66a68d1e665d17db in_seq=0 out_seq=0 2024-05-10T21:12:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.909+0000 7f2d03e00640 10 client.26126 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.909+0000 7f2d03e00640 1 -- 172.21.15.50:0/633139821 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5e2155160 con 0x55a5e1dd51c0 2024-05-10T21:12:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.909+0000 7f2d03e00640 10 client.26126 handle_command_reply: tid=0 2024-05-10T21:12:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.909+0000 7f2d30433740 1 client.26126 shutdown 2024-05-10T21:12:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.918+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 msgr2=0x55a5e21f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.918+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005280 tx=0x7f2d140052b0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 shutdown_connections 2024-05-10T21:12:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e1dd51c0 0x55a5e21fa880 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1dffd80 0x55a5e21f4400 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e411b0 0x55a5e21ed4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 --2- 172.21.15.50:0/633139821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e204d0 0x55a5e21ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.922+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 >> 172.21.15.50:0/633139821 conn(0x55a5e1e35300 msgr2=0x55a5e21e1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.929+0000 7f2d03e00640 0 client.26126 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.933+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 shutdown_connections 2024-05-10T21:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.937+0000 7f2d30433740 1 -- 172.21.15.50:0/633139821 wait complete. 2024-05-10T21:12:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:42.937+0000 7f2d30433740 20 client.26126 trim_cache size 0 max 16384 2024-05-10T21:12:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.076+0000 7f2d30433740 1 Processor -- start 2024-05-10T21:12:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.079+0000 7f2d30433740 1 -- start start 2024-05-10T21:12:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.084+0000 7f2d30433740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.084+0000 7f2d30433740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 0x55a5e21e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.085+0000 7f2d21e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 0x55a5e21e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.085+0000 7f2d22800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.085+0000 7f2d21e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 0x55a5e21e79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59312/0 (socket says 172.21.15.50:59312) 2024-05-10T21:12:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.085+0000 7f2d22800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44624/0 (socket says 172.21.15.50:44624) 2024-05-10T21:12:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.085+0000 7f2d30433740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e411b0 0x55a5e21f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.086+0000 7f2d21e00640 1 -- 172.21.15.50:0/3619809924 learned_addr learned my addr 172.21.15.50:0/3619809924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.086+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241bc500 con 0x55a5e1dffd80 2024-05-10T21:12:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.086+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b9d10 con 0x55a5e1e204d0 2024-05-10T21:12:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.086+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b7b80 con 0x55a5e1e411b0 2024-05-10T21:12:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.086+0000 7f2d23200640 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e411b0 0x55a5e21f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.088+0000 7f2d22800640 1 -- 172.21.15.50:0/3619809924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e411b0 msgr2=0x55a5e21f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.088+0000 7f2d22800640 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e411b0 0x55a5e21f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.088+0000 7f2d22800640 1 -- 172.21.15.50:0/3619809924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 msgr2=0x55a5e21e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.088+0000 7f2d22800640 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 0x55a5e21e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.088+0000 7f2d22800640 1 -- 172.21.15.50:0/3619809924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e21da470 con 0x55a5e1dffd80 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.089+0000 7f2d22800640 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a5e21e44c0 tx=0x7f2d10046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9cfbb81834b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.092+0000 7f2d0ea00640 1 -- 172.21.15.50:0/3619809924 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10076c70 con 0x55a5e1dffd80 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.092+0000 7f2d0ea00640 1 -- 172.21.15.50:0/3619809924 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d100ca360 con 0x55a5e1dffd80 2024-05-10T21:12:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.092+0000 7f2d0ea00640 1 -- 172.21.15.50:0/3619809924 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d100c5ef0 con 0x55a5e1dffd80 2024-05-10T21:12:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.097+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 msgr2=0x55a5e21dedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.097+0000 7f2d30433740 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a5e21e44c0 tx=0x7f2d10046040 comp rx=0 tx=0).stop 2024-05-10T21:12:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 shutdown_connections 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1e411b0 0x55a5e21f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e204d0 0x55a5e21e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 --2- 172.21.15.50:0/3619809924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1dffd80 0x55a5e21dedc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 >> 172.21.15.50:0/3619809924 conn(0x55a5e1e35300 msgr2=0x55a5e21de6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 shutdown_connections 2024-05-10T21:12:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.098+0000 7f2d30433740 1 -- 172.21.15.50:0/3619809924 wait complete. 2024-05-10T21:12:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.100+0000 7f2d30433740 1 -- start start 2024-05-10T21:12:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.102+0000 7f2d30433740 1 Processor -- start 2024-05-10T21:12:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.105+0000 7f2d30433740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.105+0000 7f2d30433740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.106+0000 7f2d22800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.106+0000 7f2d22800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:51692/0 (socket says 172.21.15.50:51692) 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.107+0000 7f2d30433740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.107+0000 7f2d30433740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b81d0 con 0x55a5e1e204d0 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.107+0000 7f2d30433740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b9640 con 0x55a5e1e411b0 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.107+0000 7f2d30433740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241b88a0 con 0x55a5e1dffd80 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.107+0000 7f2d30433740 10 client.0 _ll_register_callbacks cb 0x7f2d14014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59322/0 (socket says 172.21.15.50:59322) 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 -- 172.21.15.50:0/3906516678 learned_addr learned my addr 172.21.15.50:0/3906516678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d21e00640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 -- 172.21.15.50:0/3906516678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 msgr2=0x55a5e21f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 -- 172.21.15.50:0/3906516678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 msgr2=0x55a5e21f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.108+0000 7f2d23200640 1 -- 172.21.15.50:0/3906516678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e21f4d80 con 0x55a5e1e411b0 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.109+0000 7f2d21e00640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.109+0000 7f2d22800640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.109+0000 7f2d23200640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400f010 tx=0x7f2d1402f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394d0c40fd8b8bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.111+0000 7f2d03e00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.111+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400c7b0 con 0x55a5e1e411b0 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.111+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2d14021040 con 0x55a5e1e411b0 2024-05-10T21:12:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.111+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d100ca360 con 0x55a5e1e411b0 2024-05-10T21:12:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.118+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d1400c7b0 con 0x55a5e1e411b0 2024-05-10T21:12:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.118+0000 7f2d30433740 10 client.26132 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.118+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5e2262060 con 0x55a5e1e411b0 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.118+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d100ca360 con 0x55a5e1e411b0 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d14010940 con 0x55a5e1e411b0 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 10 client.26132 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 20 client.26132 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e22629f0 0x55a5e2264e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 4 client.26132 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:12:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.119+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5e214efd0 con 0x55a5e22629f0 2024-05-10T21:12:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.125+0000 7f2d22800640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e22629f0 0x55a5e2264e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.126+0000 7f2d22800640 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e22629f0 0x55a5e2264e10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d02821f4b513205 server_cookie=6d779ba4d93a253a in_seq=0 out_seq=0 2024-05-10T21:12:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.126+0000 7f2d03e00640 10 client.26132 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:43.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:42 smithi142 ceph-mon[34846]: pgmap v576: 97 pgs: 97 active+clean; 188 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 408 op/s 2024-05-10T21:12:43.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:42 smithi050 ceph-mon[29546]: pgmap v576: 97 pgs: 97 active+clean; 188 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 408 op/s 2024-05-10T21:12:43.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:42 smithi092 ceph-mon[34415]: pgmap v576: 97 pgs: 97 active+clean; 188 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 408 op/s 2024-05-10T21:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.405+0000 7f2d03e00640 1 -- 172.21.15.50:0/3906516678 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5e214efd0 con 0x55a5e22629f0 2024-05-10T21:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.405+0000 7f2d03e00640 10 client.26132 handle_command_reply: tid=0 2024-05-10T21:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.415+0000 7f2d30433740 1 client.26132 shutdown 2024-05-10T21:12:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.417+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 msgr2=0x55a5e2260c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.417+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400f010 tx=0x7f2d1402f8e0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 shutdown_connections 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a5e22629f0 0x55a5e2264e10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5e1e411b0 0x55a5e2260c20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5e1e204d0 0x55a5e21f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 --2- 172.21.15.50:0/3906516678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a5e1dffd80 0x55a5e21f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 >> 172.21.15.50:0/3906516678 conn(0x55a5e1e35300 msgr2=0x55a5e21de620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.420+0000 7f2d03e00640 0 client.26132 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.428+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 shutdown_connections 2024-05-10T21:12:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.430+0000 7f2d30433740 1 -- 172.21.15.50:0/3906516678 wait complete. 2024-05-10T21:12:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.431+0000 7f2d30433740 20 client.26132 trim_cache size 0 max 16384 2024-05-10T21:12:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:12:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:12:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:12:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:12:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.447+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 msgr2=0x7f2d080b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.447+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 0x7f2d080b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007350 tx=0x7f2d1c0072a0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.448+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 msgr2=0x7f2d241a8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.448+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009850 tx=0x7f2d10003f80 comp rx=0 tx=0).stop 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 shutdown_connections 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2d080b2860 0x7f2d080b4d70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d2410ffe0 0x7f2d241b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d2410b2c0 0x7f2d241a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 --2- 172.21.15.50:0/229137479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2410a470 0x7f2d241a8d50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.451+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 >> 172.21.15.50:0/229137479 conn(0x7f2d24070460 msgr2=0x7f2d24072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.456+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 shutdown_connections 2024-05-10T21:12:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:43.457+0000 7f2d28a00640 1 -- 172.21.15.50:0/229137479 wait complete. 2024-05-10T21:12:45.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:44 smithi142 ceph-mon[34846]: pgmap v577: 97 pgs: 97 active+clean; 190 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2024-05-10T21:12:45.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:44 smithi050 ceph-mon[29546]: pgmap v577: 97 pgs: 97 active+clean; 190 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2024-05-10T21:12:45.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:44 smithi092 ceph-mon[34415]: pgmap v577: 97 pgs: 97 active+clean; 190 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2024-05-10T21:12:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 -- 172.21.15.50:0/1425318662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105310 msgr2=0x7fc8e0105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/1425318662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105310 0x7fc8e0105720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009990 tx=0x7fc8d0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 -- 172.21.15.50:0/1425318662 shutdown_connections 2024-05-10T21:12:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/1425318662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e010e740 0x7fc8e0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/1425318662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105310 0x7fc8e0105720 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/1425318662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105c60 0x7fc8e010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.532+0000 7fc8dfe00700 1 -- 172.21.15.50:0/1425318662 >> 172.21.15.50:0/1425318662 conn(0x7fc8e0078ac0 msgr2=0x7fc8e0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:45.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.533+0000 7fc8dfe00700 1 -- 172.21.15.50:0/1425318662 shutdown_connections 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.533+0000 7fc8dfe00700 1 -- 172.21.15.50:0/1425318662 wait complete. 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.535+0000 7fc8dfe00700 1 Processor -- start 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.535+0000 7fc8dfe00700 1 -- start start 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105310 0x7fc8e01a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e010e740 0x7fc8e01ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01142a0 con 0x7fc8e0105c60 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0114120 con 0x7fc8e010e740 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dfe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0114420 con 0x7fc8e0105310 2024-05-10T21:12:45.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8de000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8de000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50676/0 (socket says 172.21.15.50:50676) 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8de000700 1 -- 172.21.15.50:0/2213316398 learned_addr learned my addr 172.21.15.50:0/2213316398 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.536+0000 7fc8dea00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105310 0x7fc8e01a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 -- 172.21.15.50:0/2213316398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105310 msgr2=0x7fc8e01a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105310 0x7fc8e01a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 -- 172.21.15.50:0/2213316398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e010e740 msgr2=0x7fc8e01ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e010e740 0x7fc8e01ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 -- 172.21.15.50:0/2213316398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e01ad540 con 0x7fc8e0105c60 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.537+0000 7fc8de000700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9a0 tx=0x7fc8cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.538+0000 7fc8d7400700 1 -- 172.21.15.50:0/2213316398 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00cbd0 con 0x7fc8e0105c60 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.538+0000 7fc8d7400700 1 -- 172.21.15.50:0/2213316398 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc00cd70 con 0x7fc8e0105c60 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.538+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e01ad770 con 0x7fc8e0105c60 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.538+0000 7fc8d7400700 1 -- 172.21.15.50:0/2213316398 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc018680 con 0x7fc8e0105c60 2024-05-10T21:12:45.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.538+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01adcb0 con 0x7fc8e0105c60 2024-05-10T21:12:45.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.546+0000 7fc8f41b8b80 1 Processor -- start 2024-05-10T21:12:45.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 -- start start 2024-05-10T21:12:45.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8d7400700 1 -- 172.21.15.50:0/2213316398 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc8cc004020 con 0x7fc8e0105c60 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e851960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56216e76ebb0 con 0x56216e84d6a0 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56216e70c8f0 con 0x56216e851ea0 2024-05-10T21:12:45.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56216e76ab70 con 0x56216e6cefd0 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.547+0000 7fc8df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54398/0 (socket says 172.21.15.50:54398) 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 -- 172.21.15.50:0/1120232295 learned_addr learned my addr 172.21.15.50:0/1120232295 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8dea00700 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e851960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 -- 172.21.15.50:0/1120232295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 msgr2=0x56216e8522b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8522b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 -- 172.21.15.50:0/1120232295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 msgr2=0x56216e851960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e851960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.548+0000 7fc8df400700 1 -- 172.21.15.50:0/1120232295 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216e6bba60 con 0x56216e6cefd0 2024-05-10T21:12:45.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8dea00700 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e851960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:45.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8df400700 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e3e0 tx=0x7fc8d800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c815d60202d6404e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8d7400700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 0x7fc8c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8cbe00700 1 -- 172.21.15.50:0/1120232295 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8004bb0 con 0x56216e6cefd0 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8cbe00700 1 -- 172.21.15.50:0/1120232295 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d8004d50 con 0x56216e6cefd0 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8cbe00700 1 -- 172.21.15.50:0/1120232295 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80044a0 con 0x56216e6cefd0 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8d7400700 1 -- 172.21.15.50:0/2213316398 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fc8cc0e22c0 con 0x7fc8e0105c60 2024-05-10T21:12:45.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.549+0000 7fc8dea00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 0x7fc8c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/1120232295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 msgr2=0x56216e84d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8dea00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 0x7fc8c40b4e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009750 tx=0x7fc8d000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e3e0 tx=0x7fc8d800b040 comp rx=0 tx=0).stop 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/1120232295 shutdown_connections 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84d160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/1120232295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e851960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/1120232295 >> 172.21.15.50:0/1120232295 conn(0x56216e70dc10 msgr2=0x56216e847e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/1120232295 shutdown_connections 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.551+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/1120232295 wait complete. 2024-05-10T21:12:45.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.552+0000 7fc8f41b8b80 1 -- start start 2024-05-10T21:12:45.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.552+0000 7fc8f41b8b80 1 Processor -- start 2024-05-10T21:12:45.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e857160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e851ea0 0x56216e8576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56216e859df0 con 0x56216e84d6a0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56216e6d4cb0 con 0x56216e6cefd0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56216e859f70 con 0x56216e851ea0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.553+0000 7fc8f41b8b80 10 client.0 _ll_register_callbacks cb 0x56216e702510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8de000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e857160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59180/0 (socket says 172.21.15.50:59180) 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 -- 172.21.15.50:0/3571532835 learned_addr learned my addr 172.21.15.50:0/3571532835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8df400700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e851ea0 0x56216e8576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 -- 172.21.15.50:0/3571532835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e851ea0 msgr2=0x56216e8576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e851ea0 0x56216e8576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 -- 172.21.15.50:0/3571532835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 msgr2=0x56216e857160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e857160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.554+0000 7fc8dea00700 1 -- 172.21.15.50:0/3571532835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216e7345b0 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.555+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009990 tx=0x7fc8d0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4fa14f5a9076c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.555+0000 7fc8c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.555+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0003a90 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.555+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fc8d0003c30 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.555+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8004d50 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.556+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8d000f460 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.556+0000 7fc8f41b8b80 10 client.26138 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.556+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56216e864a70 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.556+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d8004d50 con 0x56216e6cefd0 2024-05-10T21:12:45.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc8d002e070 con 0x56216e6cefd0 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 10 client.26138 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 20 client.26138 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e867050 0x56216e8694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 4 client.26138 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216e819200 con 0x56216e867050 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.557+0000 7fc8de000700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e867050 0x56216e8694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.558+0000 7fc8de000700 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e867050 0x56216e8694d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9147daefc79a13 server_cookie=e6ac48bcd70c4e9 in_seq=0 out_seq=0 2024-05-10T21:12:45.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.558+0000 7fc8c8c00700 10 client.26138 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3571532835 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56216e819200 con 0x56216e867050 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8c8c00700 10 client.26138 handle_command_reply: tid=0 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 client.26138 shutdown 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 msgr2=0x56216e856c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009990 tx=0x7fc8d0009ca0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 shutdown_connections 2024-05-10T21:12:45.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e867050 0x56216e8694d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e6cefd0 0x56216e856c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e851ea0 0x56216e8576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3571532835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e857160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 >> 172.21.15.50:0/3571532835 conn(0x56216e70dc10 msgr2=0x56216e847ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8c8c00700 0 client.26138 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 shutdown_connections 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.559+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3571532835 wait complete. 2024-05-10T21:12:45.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.560+0000 7fc8f41b8b80 20 client.26138 trim_cache size 0 max 16384 2024-05-10T21:12:45.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.603+0000 7fc8f41b8b80 1 Processor -- start 2024-05-10T21:12:45.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 -- start start 2024-05-10T21:12:45.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e8574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b79a0 con 0x56216e84d6a0 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b5170 con 0x56216e851ea0 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.604+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b2fe0 con 0x56216e6cefd0 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.605+0000 7fc8dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.605+0000 7fc8dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54420/0 (socket says 172.21.15.50:54420) 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.605+0000 7fc8dea00700 1 -- 172.21.15.50:0/3216703217 learned_addr learned my addr 172.21.15.50:0/3216703217 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.605+0000 7fc8de000700 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e8574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.605+0000 7fc8df400700 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.606+0000 7fc8dea00700 1 -- 172.21.15.50:0/3216703217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 msgr2=0x56216e8589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.606+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.606+0000 7fc8dea00700 1 -- 172.21.15.50:0/3216703217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 msgr2=0x56216e8574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.606+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e8574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.607+0000 7fc8dea00700 1 -- 172.21.15.50:0/3216703217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216e848d50 con 0x56216e6cefd0 2024-05-10T21:12:45.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.608+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005010 tx=0x7fc8d002f3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd6636c2ad9b282 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.608+0000 7fc8cb400700 1 -- 172.21.15.50:0/3216703217 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00309d0 con 0x56216e6cefd0 2024-05-10T21:12:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.608+0000 7fc8cb400700 1 -- 172.21.15.50:0/3216703217 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d0030b70 con 0x56216e6cefd0 2024-05-10T21:12:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.608+0000 7fc8cb400700 1 -- 172.21.15.50:0/3216703217 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d003b470 con 0x56216e6cefd0 2024-05-10T21:12:45.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.609+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3216703217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 msgr2=0x56216e84c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:45.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.609+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0005010 tx=0x7fc8d002f3e0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.609+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3216703217 shutdown_connections 2024-05-10T21:12:45.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.609+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.609+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e6cefd0 0x56216e84c8c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.610+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3216703217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e84d6a0 0x56216e8574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.610+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3216703217 >> 172.21.15.50:0/3216703217 conn(0x56216e70dc10 msgr2=0x56216e8582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:45.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.610+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3216703217 shutdown_connections 2024-05-10T21:12:45.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.610+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3216703217 wait complete. 2024-05-10T21:12:45.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.610+0000 7fc8f41b8b80 1 -- start start 2024-05-10T21:12:45.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.611+0000 7fc8f41b8b80 1 Processor -- start 2024-05-10T21:12:45.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 0x56216e8cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b3630 con 0x56216e6cefd0 2024-05-10T21:12:45.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b4aa0 con 0x56216e851ea0 2024-05-10T21:12:45.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e01b3d00 con 0x56216e84d6a0 2024-05-10T21:12:45.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8f41b8b80 10 client.0 _ll_register_callbacks cb 0x56216e702510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:45.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.612+0000 7fc8de000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.613+0000 7fc8de000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54426/0 (socket says 172.21.15.50:54426) 2024-05-10T21:12:45.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.613+0000 7fc8de000700 1 -- 172.21.15.50:0/3478539157 learned_addr learned my addr 172.21.15.50:0/3478539157 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.613+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 0x56216e8cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.613+0000 7fc8df400700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.614+0000 7fc8de000700 1 -- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 msgr2=0x56216e8cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.614+0000 7fc8de000700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.614+0000 7fc8de000700 1 -- 172.21.15.50:0/3478539157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 msgr2=0x56216e8cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:45.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.614+0000 7fc8de000700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 0x56216e8cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:45.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.615+0000 7fc8de000700 1 -- 172.21.15.50:0/3478539157 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216e8cc2c0 con 0x56216e84d6a0 2024-05-10T21:12:45.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.615+0000 7fc8df400700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:45.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.615+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 0x56216e8cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:12:45.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.616+0000 7fc8de000700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc01e000 tx=0x7fc8cc0a9330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d856b04fe54b5586 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:45.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.616+0000 7fc8c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:45.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.616+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0aa010 con 0x56216e84d6a0 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.616+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fc8cc004310 con 0x56216e84d6a0 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.617+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d0030b70 con 0x56216e84d6a0 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.617+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8cc004510 con 0x56216e84d6a0 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.617+0000 7fc8f41b8b80 10 client.26098 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.617+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56216e85efe0 con 0x56216e84d6a0 2024-05-10T21:12:45.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.618+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d0030b70 con 0x56216e84d6a0 2024-05-10T21:12:45.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc8cc0a9b70 con 0x56216e84d6a0 2024-05-10T21:12:45.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:45.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 10 client.26098 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:12:45.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 20 client.26098 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:45.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e85f6a0 0x56216e8e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:45.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 4 client.26098 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}] 2024-05-10T21:12:45.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.619+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}) -- 0x56216e851cb0 con 0x56216e85f6a0 2024-05-10T21:12:45.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.620+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e85f6a0 0x56216e8e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:45.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.621+0000 7fc8c8c00700 10 client.26098 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:45.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:45.621+0000 7fc8dea00700 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e85f6a0 0x56216e8e6870 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc929f4289e3df57 server_cookie=afe34499a3cf227a in_seq=0 out_seq=0 2024-05-10T21:12:46.987 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:46 smithi050 ceph-mon[29546]: pgmap v578: 97 pgs: 97 active+clean; 190 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 -- 172.21.15.50:0/645406994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71201034c0 msgr2=0x7f71201038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 --2- 172.21.15.50:0/645406994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71201034c0 0x7f71201038d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f710800b9a0 tx=0x7f710800bcb0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 -- 172.21.15.50:0/645406994 shutdown_connections 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 --2- 172.21.15.50:0/645406994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120108740 0x7f71201100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 --2- 172.21.15.50:0/645406994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71201034c0 0x7f71201038d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 --2- 172.21.15.50:0/645406994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7120107df0 0x7f7120108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 -- 172.21.15.50:0/645406994 >> 172.21.15.50:0/645406994 conn(0x7f71200fedc0 msgr2=0x7f71201011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.274+0000 7f7126200700 1 -- 172.21.15.50:0/645406994 shutdown_connections 2024-05-10T21:12:47.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.275+0000 7f7126200700 1 -- 172.21.15.50:0/645406994 wait complete. 2024-05-10T21:12:47.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.277+0000 7f7126200700 1 Processor -- start 2024-05-10T21:12:47.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.277+0000 7f7126200700 1 -- start start 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.277+0000 7f7126200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71201034c0 0x7f712019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.278+0000 7f7126200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120107df0 0x7f7120198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.278+0000 7f7126200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.278+0000 7f7126200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112320 con 0x7f7120108740 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.278+0000 7f7126200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71201121a0 con 0x7f7120107df0 2024-05-10T21:12:47.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.278+0000 7f7126200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71201124a0 con 0x7f71201034c0 2024-05-10T21:12:47.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.279+0000 7f711fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.279+0000 7f711fe00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50734/0 (socket says 172.21.15.50:50734) 2024-05-10T21:12:47.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.279+0000 7f711fe00700 1 -- 172.21.15.50:0/3968227619 learned_addr learned my addr 172.21.15.50:0/3968227619 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:47.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.279+0000 7f711f400700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71201034c0 0x7f712019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.280+0000 7f711fe00700 1 -- 172.21.15.50:0/3968227619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71201034c0 msgr2=0x7f712019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.280+0000 7f711fe00700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71201034c0 0x7f712019f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.280+0000 7f711fe00700 1 -- 172.21.15.50:0/3968227619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120107df0 msgr2=0x7f7120198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.280+0000 7f711fe00700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120107df0 0x7f7120198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.280+0000 7f711fe00700 1 -- 172.21.15.50:0/3968227619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712019d480 con 0x7f7120108740 2024-05-10T21:12:47.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.281+0000 7f711fe00700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f711800ef30 tx=0x7f711800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.282+0000 7f711cc00700 1 -- 172.21.15.50:0/3968227619 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180092e0 con 0x7f7120108740 2024-05-10T21:12:47.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.282+0000 7f711cc00700 1 -- 172.21.15.50:0/3968227619 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118004bb0 con 0x7f7120108740 2024-05-10T21:12:47.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.282+0000 7f711cc00700 1 -- 172.21.15.50:0/3968227619 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004280 con 0x7f7120108740 2024-05-10T21:12:47.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.283+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712019d710 con 0x7f7120108740 2024-05-10T21:12:47.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.283+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712019dc00 con 0x7f7120108740 2024-05-10T21:12:47.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.288+0000 7f7134d13b80 1 Processor -- start 2024-05-10T21:12:47.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.289+0000 7f711cc00700 1 -- 172.21.15.50:0/3968227619 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f711801b030 con 0x7f7120108740 2024-05-10T21:12:47.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.290+0000 7f7134d13b80 1 -- start start 2024-05-10T21:12:47.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc6d4a20 con 0x55c0cc634e40 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc672760 con 0x55c0cc7b8830 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.291+0000 7f7134d13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc6d09e0 con 0x55c0cc7b29f0 2024-05-10T21:12:47.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.292+0000 7f711f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.292+0000 7f711f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59238/0 (socket says 172.21.15.50:59238) 2024-05-10T21:12:47.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.292+0000 7f711f400700 1 -- 172.21.15.50:0/3631310347 learned_addr learned my addr 172.21.15.50:0/3631310347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:47.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.292+0000 7f7116a00700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.292+0000 7f7116a00700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.293+0000 7f7116a00700 1 -- 172.21.15.50:0/3631310347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 msgr2=0x55c0cc7b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:47.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.293+0000 7f7116a00700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.293+0000 7f7116a00700 1 -- 172.21.15.50:0/3631310347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 msgr2=0x55c0cc7b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.293+0000 7f7116a00700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.294+0000 7f7116a00700 1 -- 172.21.15.50:0/3631310347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0cc6218d0 con 0x55c0cc634e40 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.294+0000 7f711f400700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.294+0000 7f711cc00700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 0x7f71000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.294+0000 7f7116a00700 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7b82f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7120102440 tx=0x7f711000b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935ee8ec2aa2bca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.295+0000 7f711f400700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 0x7f71000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.295+0000 7f711cc00700 1 -- 172.21.15.50:0/3968227619 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f71180e1db0 con 0x7f7120108740 2024-05-10T21:12:47.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.295+0000 7f7114c00700 1 -- 172.21.15.50:0/3631310347 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004220 con 0x55c0cc634e40 2024-05-10T21:12:47.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.295+0000 7f7114c00700 1 -- 172.21.15.50:0/3631310347 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100043c0 con 0x55c0cc634e40 2024-05-10T21:12:47.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.295+0000 7f7114c00700 1 -- 172.21.15.50:0/3631310347 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110005650 con 0x55c0cc634e40 2024-05-10T21:12:47.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.296+0000 7f7134d13b80 1 -- 172.21.15.50:0/3631310347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 msgr2=0x55c0cc7b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.296+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7b82f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7120102440 tx=0x7f711000b350 comp rx=0 tx=0).stop 2024-05-10T21:12:47.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.297+0000 7f7134d13b80 1 -- 172.21.15.50:0/3631310347 shutdown_connections 2024-05-10T21:12:47.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.297+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.297+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7b82f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.297+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3631310347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.297+0000 7f7134d13b80 1 -- 172.21.15.50:0/3631310347 >> 172.21.15.50:0/3631310347 conn(0x55c0cc679f70 msgr2=0x55c0cc7ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.298+0000 7f711f400700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 0x7f71000b4e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55c0cc7aeab0 tx=0x7f710800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.298+0000 7f7134d13b80 1 -- 172.21.15.50:0/3631310347 shutdown_connections 2024-05-10T21:12:47.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.299+0000 7f7134d13b80 1 -- 172.21.15.50:0/3631310347 wait complete. 2024-05-10T21:12:47.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.299+0000 7f7134d13b80 1 -- start start 2024-05-10T21:12:47.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.300+0000 7f7134d13b80 1 Processor -- start 2024-05-10T21:12:47.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.300+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.300+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.300+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f7134d13b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc7bf2b0 con 0x55c0cc634e40 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f7134d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc63ab20 con 0x55c0cc7b29f0 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f7134d13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c0cc7bf430 con 0x55c0cc7b8830 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f7134d13b80 10 client.0 _ll_register_callbacks cb 0x55c0cc67bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:47.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f711fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f711fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54444/0 (socket says 172.21.15.50:54444) 2024-05-10T21:12:47.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.301+0000 7f711fe00700 1 -- 172.21.15.50:0/3629784951 learned_addr learned my addr 172.21.15.50:0/3629784951 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:47.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711f400700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 -- 172.21.15.50:0/3629784951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b29f0 msgr2=0x55c0cc7bc4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:12:47.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7bc4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 -- 172.21.15.50:0/3629784951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 msgr2=0x55c0cc7bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:47.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7bbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 -- 172.21.15.50:0/3629784951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0cc69a420 con 0x55c0cc7b8830 2024-05-10T21:12:47.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f711fe00700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f71180a9190 tx=0x7f71180a99d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ca1f1732576e33 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f710d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:47.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180a9b90 con 0x55c0cc7b8830 2024-05-10T21:12:47.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71100043c0 con 0x55c0cc7b8830 2024-05-10T21:12:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.302+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f71180a3bb0 con 0x55c0cc7b8830 2024-05-10T21:12:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71180a9490 con 0x55c0cc7b8830 2024-05-10T21:12:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 10 client.26110 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:47.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0cc7c9e30 con 0x55c0cc7b8830 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100043c0 con 0x55c0cc7b8830 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f711800cb30 con 0x55c0cc7b8830 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 10 client.26110 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 10 client.26110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 20 client.26110 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.303+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc7cc3c0 0x55c0cc7ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.304+0000 7f7134d13b80 4 client.26110 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.304+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0cc77f1c0 con 0x55c0cc7cc3c0 2024-05-10T21:12:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.305+0000 7f711f400700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc7cc3c0 0x55c0cc7ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.305+0000 7f711f400700 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc7cc3c0 0x55c0cc7ce7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8f36697e3df3106 server_cookie=862073d160a98e2 in_seq=0 out_seq=0 2024-05-10T21:12:47.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.305+0000 7f710d600700 10 client.26110 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f710d600700 1 -- 172.21.15.50:0/3629784951 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0cc77f1c0 con 0x55c0cc7cc3c0 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f710d600700 10 client.26110 handle_command_reply: tid=0 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 client.26110 shutdown 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 msgr2=0x55c0cc7bca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f71180a9190 tx=0x7f71180a99d0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 shutdown_connections 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc7cc3c0 0x55c0cc7ce7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b8830 0x55c0cc7bca00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc634e40 0x55c0cc7bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 --2- 172.21.15.50:0/3629784951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f710d600700 0 client.26110 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 >> 172.21.15.50:0/3629784951 conn(0x55c0cc679f70 msgr2=0x55c0cc7b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 shutdown_connections 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.307+0000 7f7134d13b80 1 -- 172.21.15.50:0/3629784951 wait complete. 2024-05-10T21:12:47.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.308+0000 7f7134d13b80 20 client.26110 trim_cache size 0 max 16384 2024-05-10T21:12:47.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.343+0000 7f7134d13b80 1 Processor -- start 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.343+0000 7f7134d13b80 1 -- start start 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71201c8b00 con 0x55c0cc7b8830 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7120073cd0 con 0x55c0cc634e40 2024-05-10T21:12:47.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f7134d13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7120071b80 con 0x55c0cc7b29f0 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f711f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f711f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:59254/0 (socket says 172.21.15.50:59254) 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f711f400700 1 -- 172.21.15.50:0/2431428086 learned_addr learned my addr 172.21.15.50:0/2431428086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.344+0000 7f711fe00700 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 -- 172.21.15.50:0/2431428086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 msgr2=0x55c0cc7ad340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7ad340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 -- 172.21.15.50:0/2431428086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 msgr2=0x55c0cc7b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:47.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 -- 172.21.15.50:0/2431428086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0cc7bbd50 con 0x55c0cc634e40 2024-05-10T21:12:47.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f711f400700 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7108000c00 tx=0x7f7108020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183d021139b1e813 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.345+0000 7f710fe00700 1 -- 172.21.15.50:0/2431428086 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108028040 con 0x55c0cc634e40 2024-05-10T21:12:47.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.346+0000 7f710fe00700 1 -- 172.21.15.50:0/2431428086 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710803b070 con 0x55c0cc634e40 2024-05-10T21:12:47.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.346+0000 7f710fe00700 1 -- 172.21.15.50:0/2431428086 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108018690 con 0x55c0cc634e40 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 -- 172.21.15.50:0/2431428086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 msgr2=0x55c0cc7b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7108000c00 tx=0x7f7108020000 comp rx=0 tx=0).stop 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 -- 172.21.15.50:0/2431428086 shutdown_connections 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc634e40 0x55c0cc7b24b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 --2- 172.21.15.50:0/2431428086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 -- 172.21.15.50:0/2431428086 >> 172.21.15.50:0/2431428086 conn(0x55c0cc679f70 msgr2=0x55c0cc7b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 -- 172.21.15.50:0/2431428086 shutdown_connections 2024-05-10T21:12:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.347+0000 7f7134d13b80 1 -- 172.21.15.50:0/2431428086 wait complete. 2024-05-10T21:12:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.348+0000 7f7134d13b80 1 -- start start 2024-05-10T21:12:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.348+0000 7f7134d13b80 1 Processor -- start 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7c4e10 0x55c0cc7d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71200721d0 con 0x55c0cc7b8830 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7120073600 con 0x55c0cc7c4e10 2024-05-10T21:12:47.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7120072860 con 0x55c0cc7b29f0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f7134d13b80 10 client.0 _ll_register_callbacks cb 0x55c0cc67bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.349+0000 7f711f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54452/0 (socket says 172.21.15.50:54452) 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 -- 172.21.15.50:0/36304997 learned_addr learned my addr 172.21.15.50:0/36304997 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711fe00700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7c4e10 0x55c0cc7d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 -- 172.21.15.50:0/36304997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7c4e10 msgr2=0x55c0cc7d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7c4e10 0x55c0cc7d35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 -- 172.21.15.50:0/36304997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 msgr2=0x55c0cc7d30b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7d30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.350+0000 7f711f400700 1 -- 172.21.15.50:0/36304997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0cc7d3d60 con 0x55c0cc7b29f0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.351+0000 7f711f400700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7108005ef0 tx=0x7f7108010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2796c84f9cee43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.351+0000 7f710d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.351+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108028050 con 0x55c0cc7b29f0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.351+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7108005250 con 0x55c0cc7b29f0 2024-05-10T21:12:47.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.351+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f710803b070 con 0x55c0cc7b29f0 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.352+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7108009930 con 0x55c0cc7b29f0 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.352+0000 7f7134d13b80 10 client.26113 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.352+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0cc836a10 con 0x55c0cc7b29f0 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.352+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710803b070 con 0x55c0cc7b29f0 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7108003680 con 0x55c0cc7b29f0 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 10 client.26113 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 10 client.26113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:12:47.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 20 client.26113 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc837020 0x55c0cc839440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 4 client.26113 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.353+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0cc6e2400 con 0x55c0cc837020 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.354+0000 7f7116a00700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc837020 0x55c0cc839440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.355+0000 7f710d600700 10 client.26113 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:12:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.355+0000 7f7116a00700 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc837020 0x55c0cc839440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a2df5abd600c6b2 server_cookie=3b4182062679a976 in_seq=0 out_seq=0 2024-05-10T21:12:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.356+0000 7f710d600700 1 -- 172.21.15.50:0/36304997 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0cc6e2400 con 0x55c0cc837020 2024-05-10T21:12:47.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.356+0000 7f710d600700 10 client.26113 handle_command_reply: tid=0 2024-05-10T21:12:47.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.356+0000 7f7134d13b80 1 client.26113 shutdown 2024-05-10T21:12:47.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.357+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 msgr2=0x55c0cc7c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.357+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7108005ef0 tx=0x7f7108010a90 comp rx=0 tx=0).stop 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 shutdown_connections 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55c0cc837020 0x55c0cc839440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0cc7c4e10 0x55c0cc7d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0cc7b29f0 0x55c0cc7c4880 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 --2- 172.21.15.50:0/36304997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0cc7b8830 0x55c0cc7d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 >> 172.21.15.50:0/36304997 conn(0x55c0cc679f70 msgr2=0x55c0cc7a70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.359+0000 7f710d600700 0 client.26113 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:12:47.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.360+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 shutdown_connections 2024-05-10T21:12:47.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.360+0000 7f7134d13b80 1 -- 172.21.15.50:0/36304997 wait complete. 2024-05-10T21:12:47.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.360+0000 7f7134d13b80 20 client.26113 trim_cache size 0 max 16384 2024-05-10T21:12:47.358 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:12:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 msgr2=0x7f71000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 0x7f71000b4e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55c0cc7aeab0 tx=0x7f710800d040 comp rx=0 tx=0).stop 2024-05-10T21:12:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 msgr2=0x7f71201990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f711800ef30 tx=0x7f711800c450 comp rx=0 tx=0).stop 2024-05-10T21:12:47.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 shutdown_connections 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71000b2900 0x7f71000b4e10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7120108740 0x7f71201990d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71201034c0 0x7f712019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 --2- 172.21.15.50:0/3968227619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120107df0 0x7f7120198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 >> 172.21.15.50:0/3968227619 conn(0x7f71200fedc0 msgr2=0x7f71201096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 shutdown_connections 2024-05-10T21:12:47.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:47.361+0000 7f7126200700 1 -- 172.21.15.50:0/3968227619 wait complete. 2024-05-10T21:12:47.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:46 smithi092 ceph-mon[34415]: pgmap v578: 97 pgs: 97 active+clean; 190 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-10T21:12:47.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:46 smithi142 ceph-mon[34846]: pgmap v578: 97 pgs: 97 active+clean; 190 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-05-10T21:12:48.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:12:48.794 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:49.156 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:48 smithi050 ceph-mon[29546]: pgmap v579: 97 pgs: 97 active+clean; 198 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 298 op/s 2024-05-10T21:12:49.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:48 smithi092 ceph-mon[34415]: pgmap v579: 97 pgs: 97 active+clean; 198 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 298 op/s 2024-05-10T21:12:49.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:48 smithi142 ceph-mon[34846]: pgmap v579: 97 pgs: 97 active+clean; 198 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 298 op/s 2024-05-10T21:12:51.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:50 smithi050 ceph-mon[29546]: pgmap v580: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-05-10T21:12:51.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:50 smithi092 ceph-mon[34415]: pgmap v580: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-05-10T21:12:51.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:50 smithi142 ceph-mon[34846]: pgmap v580: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 317 op/s 2024-05-10T21:12:51.381 INFO:tasks.workunit.client.0.smithi050.stdout: 54 179946 129 112578 28 65741 57 2024-05-10T21:12:52.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.990+0000 7fc8c8c00700 1 -- 172.21.15.50:0/3478539157 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56216e851cb0 con 0x56216e85f6a0 2024-05-10T21:12:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.990+0000 7fc8c8c00700 10 client.26098 handle_command_reply: tid=0 2024-05-10T21:12:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.990+0000 7fc8f41b8b80 1 client.26098 shutdown 2024-05-10T21:12:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 msgr2=0x56216e8cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc01e000 tx=0x7fc8cc0a9330 comp rx=0 tx=0).stop 2024-05-10T21:12:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 shutdown_connections 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56216e85f6a0 0x56216e8e6870 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216e851ea0 0x56216e8cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56216e6cefd0 0x56216e8cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 --2- 172.21.15.50:0/3478539157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216e84d6a0 0x56216e8cb810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8c8c00700 0 client.26098 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 >> 172.21.15.50:0/3478539157 conn(0x56216e70dc10 msgr2=0x56216e84c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.991+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 shutdown_connections 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.992+0000 7fc8f41b8b80 1 -- 172.21.15.50:0/3478539157 wait complete. 2024-05-10T21:12:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.992+0000 7fc8f41b8b80 20 client.26098 trim_cache size 0 max 16384 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 31, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 10, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "67c3b3df": { 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 10, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:12:52.991 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 85.2, 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:12:52.992 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.993+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 msgr2=0x7fc8c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.993+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 0x7fc8c40b4e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009750 tx=0x7fc8d000b260 comp rx=0 tx=0).stop 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.993+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 msgr2=0x7fc8e01a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.993+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9a0 tx=0x7fc8cc00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 shutdown_connections 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc8c40b2900 0x7fc8c40b4e10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8e010e740 0x7fc8e01ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e0105310 0x7fc8e01a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 --2- 172.21.15.50:0/2213316398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e0105c60 0x7fc8e01a24c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:52.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 >> 172.21.15.50:0/2213316398 conn(0x7fc8e0078ac0 msgr2=0x7fc8e0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:52.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 shutdown_connections 2024-05-10T21:12:52.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:52.994+0000 7fc8dfe00700 1 -- 172.21.15.50:0/2213316398 wait complete. 2024-05-10T21:12:53.117 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:52 smithi050 ceph-mon[29546]: pgmap v581: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-10T21:12:53.117 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:52 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:12:53.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:52 smithi092 ceph-mon[34415]: pgmap v581: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-10T21:12:53.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:52 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:12:53.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:52 smithi142 ceph-mon[34846]: pgmap v581: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-10T21:12:53.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:52 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:12:54.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:12:54.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:53 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:12:54.322 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '67c3b3df', quiesce took 28.8 seconds. Will release after: 25.15 2024-05-10T21:12:54.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:12:54.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:12:54.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:53 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:12:54.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:12:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:53 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:12:54.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 -- 172.21.15.50:0/656767027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494105c60 msgr2=0x7ff49410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 --2- 172.21.15.50:0/656767027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494105c60 0x7ff49410e1b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff480009990 tx=0x7ff480009ca0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 -- 172.21.15.50:0/656767027 shutdown_connections 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 --2- 172.21.15.50:0/656767027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff49410e740 0x7ff494110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 --2- 172.21.15.50:0/656767027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105310 0x7ff494105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 --2- 172.21.15.50:0/656767027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494105c60 0x7ff49410e1b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 -- 172.21.15.50:0/656767027 >> 172.21.15.50:0/656767027 conn(0x7ff494078ac0 msgr2=0x7ff494078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.741+0000 7ff49ac00700 1 -- 172.21.15.50:0/656767027 shutdown_connections 2024-05-10T21:12:54.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.742+0000 7ff49ac00700 1 -- 172.21.15.50:0/656767027 wait complete. 2024-05-10T21:12:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.743+0000 7ff49ac00700 1 Processor -- start 2024-05-10T21:12:54.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.743+0000 7ff49ac00700 1 -- start start 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 0x7ff494103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff49410e740 0x7ff494103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941142b0 con 0x7ff49410e740 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff494114130 con 0x7ff494105310 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff49ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff494114430 con 0x7ff494105c60 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff499800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 0x7ff494103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff498e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff499800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 0x7ff494103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33106/0 (socket says 172.21.15.50:33106) 2024-05-10T21:12:54.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff498e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60462/0 (socket says 172.21.15.50:60462) 2024-05-10T21:12:54.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.744+0000 7ff499800700 1 -- 172.21.15.50:0/3290371106 learned_addr learned my addr 172.21.15.50:0/3290371106 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:54.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.745+0000 7ff498e00700 1 -- 172.21.15.50:0/3290371106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 msgr2=0x7ff494103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.745+0000 7ff498e00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 0x7ff494103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.745+0000 7ff498e00700 1 -- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff49410e740 msgr2=0x7ff494103a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.745+0000 7ff498e00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff49410e740 0x7ff494103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.745+0000 7ff498e00700 1 -- 172.21.15.50:0/3290371106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941a9030 con 0x7ff494105c60 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.746+0000 7ff498e00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.746+0000 7ff492000700 1 -- 172.21.15.50:0/3290371106 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480026070 con 0x7ff494105c60 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.746+0000 7ff492000700 1 -- 172.21.15.50:0/3290371106 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480021410 con 0x7ff494105c60 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.746+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941a92c0 con 0x7ff494105c60 2024-05-10T21:12:54.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.746+0000 7ff492000700 1 -- 172.21.15.50:0/3290371106 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480017650 con 0x7ff494105c60 2024-05-10T21:12:54.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.747+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941a97f0 con 0x7ff494105c60 2024-05-10T21:12:54.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.752+0000 7ff4aac56b80 1 Processor -- start 2024-05-10T21:12:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.752+0000 7ff4aac56b80 1 -- start start 2024-05-10T21:12:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39251540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a3924bc40 0x556a3924c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a3916da20 con 0x556a39251a80 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556a3910b760 con 0x556a3924bc40 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556a391699e0 con 0x556a390cde40 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff49a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39251540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff499800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff499800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42936/0 (socket says 172.21.15.50:42936) 2024-05-10T21:12:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff499800700 1 -- 172.21.15.50:0/3277564859 learned_addr learned my addr 172.21.15.50:0/3277564859 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.753+0000 7ff492000700 1 -- 172.21.15.50:0/3290371106 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff480029070 con 0x7ff494105c60 2024-05-10T21:12:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 -- 172.21.15.50:0/3277564859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 msgr2=0x556a39251540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39251540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 -- 172.21.15.50:0/3277564859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a3924bc40 msgr2=0x556a3924c050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a3924bc40 0x556a3924c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 -- 172.21.15.50:0/3277564859 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a390ba8d0 con 0x556a39251a80 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.754+0000 7ff499800700 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff48800e660 tx=0x7ff488009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd57e07df34e691 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.755+0000 7ff486a00700 1 -- 172.21.15.50:0/3277564859 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488003a00 con 0x556a39251a80 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.755+0000 7ff486a00700 1 -- 172.21.15.50:0/3277564859 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48801a070 con 0x556a39251a80 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.755+0000 7ff486a00700 1 -- 172.21.15.50:0/3277564859 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488007760 con 0x556a39251a80 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 -- 172.21.15.50:0/3277564859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 msgr2=0x556a3924b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff48800e660 tx=0x7ff488009920 comp rx=0 tx=0).stop 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 -- 172.21.15.50:0/3277564859 shutdown_connections 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a3924bc40 0x556a3924c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39251540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/3277564859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a3924b700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff4aac56b80 1 -- 172.21.15.50:0/3277564859 >> 172.21.15.50:0/3277564859 conn(0x556a39112f70 msgr2=0x556a39246400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.756+0000 7ff492000700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 0x7ff47c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.757+0000 7ff492000700 1 -- 172.21.15.50:0/3290371106 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff4800e2ba0 con 0x7ff494105c60 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.757+0000 7ff499800700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 0x7ff47c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.757+0000 7ff4aac56b80 1 -- 172.21.15.50:0/3277564859 shutdown_connections 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.757+0000 7ff4aac56b80 1 -- 172.21.15.50:0/3277564859 wait complete. 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.758+0000 7ff499800700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 0x7ff47c0b4e90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ce0 tx=0x7ff488000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.758+0000 7ff4aac56b80 1 -- start start 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.758+0000 7ff4aac56b80 1 Processor -- start 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39255240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 0x556a3926a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a39258490 con 0x556a39255cc0 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556a390d3b20 con 0x556a39251a80 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556a39258610 con 0x556a390cde40 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff4aac56b80 10 client.0 _ll_register_callbacks cb 0x556a39114c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff498e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff498e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33128/0 (socket says 172.21.15.50:33128) 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff498e00700 1 -- 172.21.15.50:0/2122046941 learned_addr learned my addr 172.21.15.50:0/2122046941 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff499800700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39255240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff49a200700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 0x556a3926a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff498e00700 1 -- 172.21.15.50:0/2122046941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 msgr2=0x556a39255240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.759+0000 7ff498e00700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39255240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff498e00700 1 -- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 msgr2=0x556a3926a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff498e00700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 0x556a3926a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff498e00700 1 -- 172.21.15.50:0/2122046941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a39133420 con 0x556a39251a80 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff49a200700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 0x556a3926a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff498e00700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4800aa1b0 tx=0x7ff4800a9de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e219af8e839acd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff47b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800aa220 con 0x556a39251a80 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff4800aa690 con 0x556a39251a80 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.760+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48801a070 con 0x556a39251a80 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4800ae5d0 con 0x556a39251a80 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 10 client.26165 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:54.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a3926bd60 con 0x556a39251a80 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48801a070 con 0x556a39251a80 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff4800ae220 con 0x556a39251a80 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 10 client.26165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 20 client.26165 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a3926e330 0x556a39270780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 4 client.26165 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.761+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a392181c0 con 0x556a3926e330 2024-05-10T21:12:54.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.763+0000 7ff499800700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a3926e330 0x556a39270780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.763+0000 7ff499800700 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a3926e330 0x556a39270780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9943f93a16312185 server_cookie=2a45b24de2642335 in_seq=0 out_seq=0 2024-05-10T21:12:54.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.763+0000 7ff47b400700 10 client.26165 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.766+0000 7ff47b400700 1 -- 172.21.15.50:0/2122046941 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a392181c0 con 0x556a3926e330 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.766+0000 7ff47b400700 10 client.26165 handle_command_reply: tid=0 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.766+0000 7ff4aac56b80 1 client.26165 shutdown 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 msgr2=0x556a39255780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4800aa1b0 tx=0x7ff4800a9de0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 shutdown_connections 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a3926e330 0x556a39270780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff47b400700 0 client.26165 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39255cc0 0x556a3926a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a39255240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/2122046941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a39255780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.767+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 >> 172.21.15.50:0/2122046941 conn(0x556a39112f70 msgr2=0x556a3924a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.768+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 shutdown_connections 2024-05-10T21:12:54.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.768+0000 7ff4aac56b80 1 -- 172.21.15.50:0/2122046941 wait complete. 2024-05-10T21:12:54.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.768+0000 7ff4aac56b80 20 client.26165 trim_cache size 0 max 16384 2024-05-10T21:12:54.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.785+0000 7ff4aac56b80 1 Processor -- start 2024-05-10T21:12:54.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.785+0000 7ff4aac56b80 1 -- start start 2024-05-10T21:12:54.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.785+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 0x556a3924b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.785+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a3924f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a39255cc0 0x556a392409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941aefb0 con 0x556a390cde40 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941ac780 con 0x556a39251a80 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff49419e770 con 0x556a39255cc0 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff499800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 0x556a3924b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff499800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 0x556a3924b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42948/0 (socket says 172.21.15.50:42948) 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff499800700 1 -- 172.21.15.50:0/1192014142 learned_addr learned my addr 172.21.15.50:0/1192014142 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:54.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff498e00700 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a3924f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.786+0000 7ff49a200700 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a39255cc0 0x556a392409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 -- 172.21.15.50:0/1192014142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a39255cc0 msgr2=0x556a392409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a39255cc0 0x556a392409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 -- 172.21.15.50:0/1192014142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 msgr2=0x556a3924b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 0x556a3924b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 -- 172.21.15.50:0/1192014142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a39242c40 con 0x556a39251a80 2024-05-10T21:12:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff498e00700 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a3924f8b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff480005b40 tx=0x7ff48000b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f70f46f03f1c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff486000700 1 -- 172.21.15.50:0/1192014142 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800aa220 con 0x556a39251a80 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff486000700 1 -- 172.21.15.50:0/1192014142 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800b2ff0 con 0x556a39251a80 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.787+0000 7ff486000700 1 -- 172.21.15.50:0/1192014142 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800e0d90 con 0x556a39251a80 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 -- 172.21.15.50:0/1192014142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 msgr2=0x556a3924f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a3924f8b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff480005b40 tx=0x7ff48000b490 comp rx=0 tx=0).stop 2024-05-10T21:12:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 -- 172.21.15.50:0/1192014142 shutdown_connections 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a39255cc0 0x556a392409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a390cde40 0x556a3924b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/1192014142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39251a80 0x556a3924f8b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 -- 172.21.15.50:0/1192014142 >> 172.21.15.50:0/1192014142 conn(0x556a39112f70 msgr2=0x556a392402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 -- 172.21.15.50:0/1192014142 shutdown_connections 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.789+0000 7ff4aac56b80 1 -- 172.21.15.50:0/1192014142 wait complete. 2024-05-10T21:12:54.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.790+0000 7ff4aac56b80 1 -- start start 2024-05-10T21:12:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.790+0000 7ff4aac56b80 1 Processor -- start 2024-05-10T21:12:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a392667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a39269660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 0x556a39269ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941aacf0 con 0x556a39251a80 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941ac0b0 con 0x556a39255cc0 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941ab310 con 0x556a390cde40 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff4aac56b80 10 client.0 _ll_register_callbacks cb 0x556a39114c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff499800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a392667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff499800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a392667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60488/0 (socket says 172.21.15.50:60488) 2024-05-10T21:12:54.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.791+0000 7ff499800700 1 -- 172.21.15.50:0/678036718 learned_addr learned my addr 172.21.15.50:0/678036718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:54.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 0x556a39269ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff498e00700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a39269660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 -- 172.21.15.50:0/678036718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 msgr2=0x556a392667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a392667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 -- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 msgr2=0x556a39269660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a39269660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.792+0000 7ff49a200700 1 -- 172.21.15.50:0/678036718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a39258370 con 0x556a39255cc0 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff49a200700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 0x556a39269ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009cc0 tx=0x7ff48c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5e7361fc5be480 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff47b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c004830 con 0x556a39255cc0 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800b2ff0 con 0x556a39255cc0 2024-05-10T21:12:54.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7ff48c0049d0 con 0x556a39255cc0 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff48c004bd0 con 0x556a39255cc0 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff4aac56b80 10 client.26171 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.793+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a392d8930 con 0x556a39255cc0 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800b2ff0 con 0x556a39255cc0 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff48c01a070 con 0x556a39255cc0 2024-05-10T21:12:54.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 10 client.26171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 20 client.26171 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a392d9080 0x556a392db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 4 client.26171 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.794+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a3917b400 con 0x556a392d9080 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.795+0000 7ff499800700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a392d9080 0x556a392db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.795+0000 7ff499800700 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a392d9080 0x556a392db4a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40c65288b590fdff server_cookie=9f4b276800e60b00 in_seq=0 out_seq=0 2024-05-10T21:12:54.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.795+0000 7ff47b400700 10 client.26171 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff47b400700 1 -- 172.21.15.50:0/678036718 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a3917b400 con 0x556a392d9080 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff47b400700 10 client.26171 handle_command_reply: tid=0 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 client.26171 shutdown 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 msgr2=0x556a39269ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 0x556a39269ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009cc0 tx=0x7ff48c003680 comp rx=0 tx=0).stop 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 shutdown_connections 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556a39255cc0 0x556a39269ba0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x556a392d9080 0x556a392db4a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff47b400700 0 client.26171 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:12:54.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a390cde40 0x556a392667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 --2- 172.21.15.50:0/678036718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a39251a80 0x556a39269660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.798+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 >> 172.21.15.50:0/678036718 conn(0x556a39112f70 msgr2=0x556a3924f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.799+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 shutdown_connections 2024-05-10T21:12:54.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.799+0000 7ff4aac56b80 1 -- 172.21.15.50:0/678036718 wait complete. 2024-05-10T21:12:54.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.799+0000 7ff4aac56b80 20 client.26171 trim_cache size 0 max 16384 2024-05-10T21:12:54.798 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:12:54.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 msgr2=0x7ff47c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 0x7ff47c0b4e90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ce0 tx=0x7ff488000f40 comp rx=0 tx=0).stop 2024-05-10T21:12:54.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 msgr2=0x7ff494103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:54.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800047b0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 shutdown_connections 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff47c0b2980 0x7ff47c0b4e90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff494105310 0x7ff494103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff49410e740 0x7ff494103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 --2- 172.21.15.50:0/3290371106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494105c60 0x7ff494103540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 >> 172.21.15.50:0/3290371106 conn(0x7ff494078ac0 msgr2=0x7ff4940fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.800+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 shutdown_connections 2024-05-10T21:12:54.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:12:54.801+0000 7ff49ac00700 1 -- 172.21.15.50:0/3290371106 wait complete. 2024-05-10T21:12:54.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:54 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-10T21:12:54.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:54 smithi050 ceph-mon[29546]: Unable to set osd_memory_target on smithi092 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-10T21:12:54.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:54 smithi050 ceph-mon[29546]: pgmap v582: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-10T21:12:54.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:54 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-05-10T21:12:54.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:54 smithi050 ceph-mon[29546]: Unable to set osd_memory_target on smithi050 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-10T21:12:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:54 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-10T21:12:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:54 smithi092 ceph-mon[34415]: Unable to set osd_memory_target on smithi092 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-10T21:12:55.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:54 smithi092 ceph-mon[34415]: pgmap v582: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-10T21:12:55.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:54 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-05-10T21:12:55.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:54 smithi092 ceph-mon[34415]: Unable to set osd_memory_target on smithi050 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-10T21:12:55.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:54 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-10T21:12:55.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:54 smithi142 ceph-mon[34846]: Unable to set osd_memory_target on smithi092 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-05-10T21:12:55.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:54 smithi142 ceph-mon[34846]: pgmap v582: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-10T21:12:55.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:54 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-05-10T21:12:55.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:54 smithi142 ceph-mon[34846]: Unable to set osd_memory_target on smithi050 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-10T21:12:55.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:12:56.398 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 -- 172.21.15.50:0/86780660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa04010bdc0 msgr2=0x7fa04010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 --2- 172.21.15.50:0/86780660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa04010bdc0 0x7fa04010e1e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa038009be0 tx=0x7fa03802a6d0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 -- 172.21.15.50:0/86780660 shutdown_connections 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 --2- 172.21.15.50:0/86780660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa04010bdc0 0x7fa04010e1e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 --2- 172.21.15.50:0/86780660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 --2- 172.21.15.50:0/86780660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa040105e80 0x7fa040106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 -- 172.21.15.50:0/86780660 >> 172.21.15.50:0/86780660 conn(0x7fa040101070 msgr2=0x7fa040103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 -- 172.21.15.50:0/86780660 shutdown_connections 2024-05-10T21:12:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.842+0000 7fa044a00640 1 -- 172.21.15.50:0/86780660 wait complete. 2024-05-10T21:12:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 Processor -- start 2024-05-10T21:12:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 -- start start 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa04010bdc0 0x7fa0401b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa040111980 con 0x7fa04010bdc0 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa040111800 con 0x7fa0401070c0 2024-05-10T21:12:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.844+0000 7fa044a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa040111b00 con 0x7fa040105e80 2024-05-10T21:12:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60508/0 (socket says 172.21.15.50:60508) 2024-05-10T21:12:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 -- 172.21.15.50:0/3825894642 learned_addr learned my addr 172.21.15.50:0/3825894642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03e800640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa04010bdc0 0x7fa0401b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03d400640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 -- 172.21.15.50:0/3825894642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 msgr2=0x7fa0401986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 -- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa04010bdc0 msgr2=0x7fa0401b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa04010bdc0 0x7fa0401b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03de00640 1 -- 172.21.15.50:0/3825894642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0401b0dc0 con 0x7fa040105e80 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.845+0000 7fa03d400640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.846+0000 7fa03de00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e980 tx=0x7fa03000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.846+0000 7fa036a00640 1 -- 172.21.15.50:0/3825894642 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000cd80 con 0x7fa040105e80 2024-05-10T21:12:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.846+0000 7fa036a00640 1 -- 172.21.15.50:0/3825894642 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0300108e0 con 0x7fa040105e80 2024-05-10T21:12:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.846+0000 7fa036a00640 1 -- 172.21.15.50:0/3825894642 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030018990 con 0x7fa040105e80 2024-05-10T21:12:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.847+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0401b0ff0 con 0x7fa040105e80 2024-05-10T21:12:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.847+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0401b1400 con 0x7fa040105e80 2024-05-10T21:12:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.850+0000 7fa04b9bf740 1 Processor -- start 2024-05-10T21:12:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 -- start start 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c5d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c80bc1550 con 0x564c808922f0 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564c80b66bc0 con 0x564c808bb1b0 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c80bbd510 con 0x564c8089a4d0 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa03e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa03e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33172/0 (socket says 172.21.15.50:33172) 2024-05-10T21:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.851+0000 7fa03e800640 1 -- 172.21.15.50:0/1925306238 learned_addr learned my addr 172.21.15.50:0/1925306238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.852+0000 7fa03d400640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.852+0000 7fa03e800640 1 -- 172.21.15.50:0/1925306238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 msgr2=0x564c80c64580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.852+0000 7fa03e800640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c64580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.852+0000 7fa03e800640 1 -- 172.21.15.50:0/1925306238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 msgr2=0x564c80c5d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.852+0000 7fa036a00640 1 -- 172.21.15.50:0/3825894642 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa03001f020 con 0x7fa040105e80 2024-05-10T21:12:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.853+0000 7fa03de00640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c5d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.853+0000 7fa03e800640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c5d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.853+0000 7fa03e800640 1 -- 172.21.15.50:0/1925306238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c808d3210 con 0x564c808bb1b0 2024-05-10T21:12:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.853+0000 7fa03de00640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c5d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.853+0000 7fa03e800640 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa038009be0 tx=0x7fa0380303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f205d0b0f1002c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.854+0000 7fa02b000640 1 -- 172.21.15.50:0/1925306238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038040030 con 0x564c808bb1b0 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.854+0000 7fa02b000640 1 -- 172.21.15.50:0/1925306238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa038030590 con 0x564c808bb1b0 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.854+0000 7fa02b000640 1 -- 172.21.15.50:0/1925306238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038036040 con 0x564c808bb1b0 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.854+0000 7fa036a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 0x7fa0240b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.854+0000 7fa036a00640 1 -- 172.21.15.50:0/3825894642 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa0300e2420 con 0x7fa040105e80 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 -- 172.21.15.50:0/1925306238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 msgr2=0x564c80c5d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa038009be0 tx=0x7fa0380303a0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa03de00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 0x7fa0240b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 -- 172.21.15.50:0/1925306238 shutdown_connections 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c64580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c5d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/1925306238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808bb1b0 0x564c80c5d100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 -- 172.21.15.50:0/1925306238 >> 172.21.15.50:0/1925306238 conn(0x564c808af300 msgr2=0x564c80c58a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.855+0000 7fa04b9bf740 1 -- 172.21.15.50:0/1925306238 shutdown_connections 2024-05-10T21:12:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.856+0000 7fa04b9bf740 1 -- 172.21.15.50:0/1925306238 wait complete. 2024-05-10T21:12:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.856+0000 7fa03de00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 0x7fa0240b4ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030018420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.856+0000 7fa04b9bf740 1 -- start start 2024-05-10T21:12:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.857+0000 7fa04b9bf740 1 Processor -- start 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.857+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c66bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.857+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c6e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c80c64f70 con 0x564c808922f0 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564c80b7a660 con 0x564c8089a4d0 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c80c650f0 con 0x564c808bb1b0 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa04b9bf740 10 client.0 _ll_register_callbacks cb 0x564c80b44400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.858+0000 7fa03e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c66bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33178/0 (socket says 172.21.15.50:33178) 2024-05-10T21:12:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 -- 172.21.15.50:0/4196220613 learned_addr learned my addr 172.21.15.50:0/4196220613 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03e800640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c6e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 -- 172.21.15.50:0/4196220613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 msgr2=0x564c80c6e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c6e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 -- 172.21.15.50:0/4196220613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 msgr2=0x564c80c66bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c66bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03d400640 1 -- 172.21.15.50:0/4196220613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c8099f620 con 0x564c8089a4d0 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.859+0000 7fa03e800640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c6e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:12:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.860+0000 7fa03d400640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa02c012670 tx=0x7fa02c012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1163385ccc86355f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.860+0000 7fa01f400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:12:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.860+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c010d60 con 0x564c8089a4d0 2024-05-10T21:12:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.860+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa02c005020 con 0x564c8089a4d0 2024-05-10T21:12:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.860+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa038030590 con 0x564c8089a4d0 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.861+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa02c005260 con 0x564c8089a4d0 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.861+0000 7fa04b9bf740 10 client.26180 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.861+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c80c71020 con 0x564c8089a4d0 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa038030590 con 0x564c8089a4d0 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa02c019030 con 0x564c8089a4d0 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 10 client.26180 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 20 client.26180 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c80879d80 0x564c80c748a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 4 client.26180 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:12:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.862+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c80bcf160 con 0x564c80879d80 2024-05-10T21:12:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.863+0000 7fa03e800640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c80879d80 0x564c80c748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.863+0000 7fa01f400640 10 client.26180 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.863+0000 7fa03e800640 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c80879d80 0x564c80c748a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6d97f53871f0d2 server_cookie=ab3c5362de1e8074 in_seq=0 out_seq=0 2024-05-10T21:12:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.864+0000 7fa01f400640 1 -- 172.21.15.50:0/4196220613 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c80bcf160 con 0x564c80879d80 2024-05-10T21:12:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.864+0000 7fa01f400640 10 client.26180 handle_command_reply: tid=0 2024-05-10T21:12:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.864+0000 7fa04b9bf740 1 client.26180 shutdown 2024-05-10T21:12:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 msgr2=0x564c80c67100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa02c012670 tx=0x7fa02c012b30 comp rx=0 tx=0).stop 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 shutdown_connections 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c80879d80 0x564c80c748a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c6e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c8089a4d0 0x564c80c67100 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa01f400640 0 client.26180 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/4196220613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808922f0 0x564c80c66bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 >> 172.21.15.50:0/4196220613 conn(0x564c808af300 msgr2=0x564c80c57a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 shutdown_connections 2024-05-10T21:12:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 1 -- 172.21.15.50:0/4196220613 wait complete. 2024-05-10T21:12:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.865+0000 7fa04b9bf740 20 client.26180 trim_cache size 0 max 16384 2024-05-10T21:12:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 Processor -- start 2024-05-10T21:12:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 -- start start 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401bca60 con 0x564c8089a4d0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401ba270 con 0x564c808922f0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.888+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b80c0 con 0x564c808bb1b0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42990/0 (socket says 172.21.15.50:42990) 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 -- 172.21.15.50:0/2191483158 learned_addr learned my addr 172.21.15.50:0/2191483158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03e800640 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 -- 172.21.15.50:0/2191483158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 msgr2=0x564c80c79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 -- 172.21.15.50:0/2191483158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 msgr2=0x564c80c5bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c5bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.889+0000 7fa03d400640 1 -- 172.21.15.50:0/2191483158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c80c54280 con 0x564c8089a4d0 2024-05-10T21:12:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.890+0000 7fa03d400640 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa02c001eb0 tx=0x7fa02c01a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3834ab7f3337cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.890+0000 7fa02a600640 1 -- 172.21.15.50:0/2191483158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0054c0 con 0x564c8089a4d0 2024-05-10T21:12:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.890+0000 7fa02a600640 1 -- 172.21.15.50:0/2191483158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c005660 con 0x564c8089a4d0 2024-05-10T21:12:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.890+0000 7fa02a600640 1 -- 172.21.15.50:0/2191483158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c024680 con 0x564c8089a4d0 2024-05-10T21:12:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 -- 172.21.15.50:0/2191483158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 msgr2=0x564c80c6df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa02c001eb0 tx=0x7fa02c01a430 comp rx=0 tx=0).stop 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 -- 172.21.15.50:0/2191483158 shutdown_connections 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c808bb1b0 0x564c80c79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c8089a4d0 0x564c80c6df80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/2191483158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.891+0000 7fa04b9bf740 1 -- 172.21.15.50:0/2191483158 >> 172.21.15.50:0/2191483158 conn(0x564c808af300 msgr2=0x564c80c5b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.892+0000 7fa04b9bf740 1 -- 172.21.15.50:0/2191483158 shutdown_connections 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.892+0000 7fa04b9bf740 1 -- 172.21.15.50:0/2191483158 wait complete. 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.892+0000 7fa04b9bf740 1 -- start start 2024-05-10T21:12:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.892+0000 7fa04b9bf740 1 Processor -- start 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c6b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 0x564c80c6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b8730 con 0x564c808bb1b0 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b9ba0 con 0x564c808922f0 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b8e00 con 0x564c8089a4d0 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa04b9bf740 10 client.0 _ll_register_callbacks cb 0x7fa02c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa03e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c6b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa03e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 0x564c80c6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa03e800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c6b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33200/0 (socket says 172.21.15.50:33200) 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.893+0000 7fa03e800640 1 -- 172.21.15.50:0/899286576 learned_addr learned my addr 172.21.15.50:0/899286576 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03d400640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c6b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 -- 172.21.15.50:0/899286576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 msgr2=0x564c80c6b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c6b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 -- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 msgr2=0x564c80c6b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c6b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 -- 172.21.15.50:0/899286576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c80c6cec0 con 0x564c808bb1b0 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa03e800640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 0x564c80c6c900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa038004af0 tx=0x7fa038033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdbbe5187ea1092 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:12:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.894+0000 7fa01f400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:12:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.895+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03804b030 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.895+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa038030a70 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.895+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa02c005660 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa038036040 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 10 client.16044 fetch_fsmap learned FSMap version 31 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c80c6e250 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c005660 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa038031440 con 0x564c808bb1b0 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 10 client.16044 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 20 client.16044 populate_metadata read hostname 'smithi050' 2024-05-10T21:12:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c8084f1c0 0x564c80cdeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:12:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 4 client.16044 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:12:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.896+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c80bc8fd0 con 0x564c8084f1c0 2024-05-10T21:12:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.897+0000 7fa03d400640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c8084f1c0 0x564c80cdeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:12:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.898+0000 7fa03d400640 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c8084f1c0 0x564c80cdeee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3879aa6b1560c128 server_cookie=ae2fdc525796a736 in_seq=0 out_seq=0 2024-05-10T21:12:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:56.898+0000 7fa01f400640 10 client.16044 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.000+0000 7fa01f400640 1 -- 172.21.15.50:0/899286576 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c80bc8fd0 con 0x564c8084f1c0 2024-05-10T21:12:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.000+0000 7fa01f400640 10 client.16044 handle_command_reply: tid=0 2024-05-10T21:12:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.002+0000 7fa04b9bf740 1 client.16044 shutdown 2024-05-10T21:12:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.002+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 msgr2=0x564c80c6c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.002+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 0x564c80c6c900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa038004af0 tx=0x7fa038033040 comp rx=0 tx=0).stop 2024-05-10T21:12:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 shutdown_connections 2024-05-10T21:12:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c8084f1c0 0x564c80cdeee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8089a4d0 0x564c80c6b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c808bb1b0 0x564c80c6c900 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 --2- 172.21.15.50:0/899286576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c808922f0 0x564c80c6b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 >> 172.21.15.50:0/899286576 conn(0x564c808af300 msgr2=0x564c80c5c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.003+0000 7fa01f400640 0 client.16044 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:12:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.004+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 shutdown_connections 2024-05-10T21:12:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.004+0000 7fa04b9bf740 1 -- 172.21.15.50:0/899286576 wait complete. 2024-05-10T21:12:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.005+0000 7fa04b9bf740 20 client.16044 trim_cache size 0 max 16384 2024-05-10T21:12:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:12:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:12:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:12:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:12:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.006+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 msgr2=0x7fa0240b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.006+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 0x7fa0240b4ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030018420 comp rx=0 tx=0).stop 2024-05-10T21:12:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.006+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 msgr2=0x7fa040198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:12:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.006+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e980 tx=0x7fa03000ee40 comp rx=0 tx=0).stop 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 shutdown_connections 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa0240b29b0 0x7fa0240b4ec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa04010bdc0 0x7fa0401b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0401070c0 0x7fa0401986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 --2- 172.21.15.50:0/3825894642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa040105e80 0x7fa040198180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:12:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.010+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 >> 172.21.15.50:0/3825894642 conn(0x7fa040101070 msgr2=0x7fa040107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:12:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.011+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 shutdown_connections 2024-05-10T21:12:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:12:57.011+0000 7fa044a00640 1 -- 172.21.15.50:0/3825894642 wait complete. 2024-05-10T21:12:57.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:56 smithi050 ceph-mon[29546]: pgmap v583: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-10T21:12:57.168 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:56 smithi092 ceph-mon[34415]: pgmap v583: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-10T21:12:57.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:56 smithi142 ceph-mon[34846]: pgmap v583: 97 pgs: 97 active+clean; 223 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-10T21:12:59.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:12:58 smithi050 ceph-mon[29546]: pgmap v584: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-05-10T21:12:59.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:12:58 smithi092 ceph-mon[34415]: pgmap v584: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-05-10T21:12:59.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:12:58 smithi142 ceph-mon[34846]: pgmap v584: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-05-10T21:13:01.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:00 smithi050 ceph-mon[29546]: pgmap v585: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-05-10T21:13:01.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.174+0000 7fb983200700 1 -- 172.21.15.50:0/582998312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 msgr2=0x7fb97c06fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.174+0000 7fb983200700 1 --2- 172.21.15.50:0/582998312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c06fb20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0098f0 tx=0x7fb96c009c00 comp rx=0 tx=0).stop 2024-05-10T21:13:01.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.175+0000 7fb983200700 1 -- 172.21.15.50:0/582998312 shutdown_connections 2024-05-10T21:13:01.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.175+0000 7fb983200700 1 --2- 172.21.15.50:0/582998312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c06fb20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.175+0000 7fb983200700 1 --2- 172.21.15.50:0/582998312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c11d9e0 0x7fb97c11fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.175+0000 7fb983200700 1 --2- 172.21.15.50:0/582998312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c070060 0x7fb97c11d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.175+0000 7fb983200700 1 -- 172.21.15.50:0/582998312 >> 172.21.15.50:0/582998312 conn(0x7fb97c10ab60 msgr2=0x7fb97c10cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.176+0000 7fb983200700 1 -- 172.21.15.50:0/582998312 shutdown_connections 2024-05-10T21:13:01.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.176+0000 7fb983200700 1 -- 172.21.15.50:0/582998312 wait complete. 2024-05-10T21:13:01.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.177+0000 7fb983200700 1 Processor -- start 2024-05-10T21:13:01.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.177+0000 7fb983200700 1 -- start start 2024-05-10T21:13:01.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 0x7fb97c1afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c11d9e0 0x7fb97c1a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1232d0 con 0x7fb97c070060 2024-05-10T21:13:01.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c123150 con 0x7fb97c06f710 2024-05-10T21:13:01.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb983200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c123450 con 0x7fb97c11d9e0 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.178+0000 7fb97be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 0x7fb97c1afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33224/0 (socket says 172.21.15.50:33224) 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 -- 172.21.15.50:0/1191963641 learned_addr learned my addr 172.21.15.50:0/1191963641 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 -- 172.21.15.50:0/1191963641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c11d9e0 msgr2=0x7fb97c1a9a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T21:13:01.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c11d9e0 0x7fb97c1a9a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 -- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 msgr2=0x7fb97c1afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 0x7fb97c1afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.179+0000 7fb97be00700 1 -- 172.21.15.50:0/1191963641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c1aa140 con 0x7fb97c06f710 2024-05-10T21:13:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.180+0000 7fb97b400700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 0x7fb97c1afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:01.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.180+0000 7fb97be00700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009fd0 tx=0x7fb96c00fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.180+0000 7fb978c00700 1 -- 172.21.15.50:0/1191963641 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c027070 con 0x7fb97c06f710 2024-05-10T21:13:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.180+0000 7fb978c00700 1 -- 172.21.15.50:0/1191963641 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c00b560 con 0x7fb97c06f710 2024-05-10T21:13:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.180+0000 7fb978c00700 1 -- 172.21.15.50:0/1191963641 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c004950 con 0x7fb97c06f710 2024-05-10T21:13:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.181+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1aa3d0 con 0x7fb97c06f710 2024-05-10T21:13:01.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.181+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1aa860 con 0x7fb97c06f710 2024-05-10T21:13:01.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.188+0000 7fb991d3fb80 1 Processor -- start 2024-05-10T21:13:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.188+0000 7fb991d3fb80 1 -- start start 2024-05-10T21:13:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.188+0000 7fb978c00700 1 -- 172.21.15.50:0/1191963641 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb96c02a070 con 0x7fb97c06f710 2024-05-10T21:13:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5636e1c0da20 con 0x5636e1cf1bf0 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636e1bab760 con 0x5636e1b6de40 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.189+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636e1c099e0 con 0x5636e1cebc50 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb97be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb981200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb97be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43016/0 (socket says 172.21.15.50:43016) 2024-05-10T21:13:01.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb97be00700 1 -- 172.21.15.50:0/1738684843 learned_addr learned my addr 172.21.15.50:0/1738684843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:01.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb97b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.190+0000 7fb978c00700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 0x7fb95c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb978c00700 1 -- 172.21.15.50:0/1191963641 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb96c0e2cf0 con 0x7fb97c06f710 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 -- 172.21.15.50:0/1738684843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 msgr2=0x5636e1cec060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cec060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 -- 172.21.15.50:0/1738684843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 msgr2=0x5636e1cf16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 -- 172.21.15.50:0/1738684843 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e1b5a8d0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb981200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 0x7fb95c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.191+0000 7fb97be00700 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a63f0 tx=0x7fb96c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be659f0989c4e762 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.193+0000 7fb971600700 1 -- 172.21.15.50:0/1738684843 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0a9cc0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.193+0000 7fb971600700 1 -- 172.21.15.50:0/1738684843 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c0e6340 con 0x5636e1cf1bf0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.193+0000 7fb971600700 1 -- 172.21.15.50:0/1738684843 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0a9ae0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.193+0000 7fb981200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 0x7fb95c0b4e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb9740079a0 tx=0x7fb974009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.194+0000 7fb991d3fb80 1 -- 172.21.15.50:0/1738684843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 msgr2=0x5636e1ceb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.194+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a63f0 tx=0x7fb96c009260 comp rx=0 tx=0).stop 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 -- 172.21.15.50:0/1738684843 shutdown_connections 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cec060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/1738684843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1ceb710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 -- 172.21.15.50:0/1738684843 >> 172.21.15.50:0/1738684843 conn(0x5636e1baca80 msgr2=0x5636e1ce6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 -- 172.21.15.50:0/1738684843 shutdown_connections 2024-05-10T21:13:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.195+0000 7fb991d3fb80 1 -- 172.21.15.50:0/1738684843 wait complete. 2024-05-10T21:13:01.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.196+0000 7fb991d3fb80 1 -- start start 2024-05-10T21:13:01.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.196+0000 7fb991d3fb80 1 Processor -- start 2024-05-10T21:13:01.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.197+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.197+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.197+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5636e1c0da20 con 0x5636e1cf1bf0 2024-05-10T21:13:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636e1b73b20 con 0x5636e1b6de40 2024-05-10T21:13:01.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636e1cf8480 con 0x5636e1cebc50 2024-05-10T21:13:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb991d3fb80 10 client.0 _ll_register_callbacks cb 0x5636e1ba1380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb97b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb97b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60584/0 (socket says 172.21.15.50:60584) 2024-05-10T21:13:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb97b400700 1 -- 172.21.15.50:0/3080742556 learned_addr learned my addr 172.21.15.50:0/3080742556 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:01.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.198+0000 7fb981200700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.199+0000 7fb97be00700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 -- 172.21.15.50:0/3080742556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 msgr2=0x5636e1cf5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 -- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 msgr2=0x5636e1cf5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 -- 172.21.15.50:0/3080742556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e1bd3420 con 0x5636e1cf1bf0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.200+0000 7fb981200700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf5ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5636e1ceca30 tx=0x7fb9740093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3835e3a1d6b143 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.201+0000 7fb96a000700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.201+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974012040 con 0x5636e1cf1bf0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.201+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb974004db0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.201+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636e1cb8070 con 0x5636e1cf1bf0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.203+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb974004250 con 0x5636e1cf1bf0 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.203+0000 7fb991d3fb80 10 client.16056 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:01.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.203+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5636e1d02f80 con 0x5636e1cf1bf0 2024-05-10T21:13:01.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.204+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5636e1cb8070 con 0x5636e1cf1bf0 2024-05-10T21:13:01.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.204+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb974021070 con 0x5636e1cf1bf0 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 10 client.16056 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 20 client.16056 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d07a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 4 client.16056 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:01.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.207+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e1c1b400 con 0x5636e1d05660 2024-05-10T21:13:01.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.208+0000 7fb97be00700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d07a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.209+0000 7fb97be00700 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d07a80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4da9be824fb8fc3 server_cookie=40f3c38783897a6 in_seq=0 out_seq=0 2024-05-10T21:13:01.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb96a000700 10 client.16056 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:01.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb96a000700 1 -- 172.21.15.50:0/3080742556 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636e1c1b400 con 0x5636e1d05660 2024-05-10T21:13:01.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb96a000700 10 client.16056 handle_command_reply: tid=0 2024-05-10T21:13:01.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb991d3fb80 1 client.16056 shutdown 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 msgr2=0x5636e1cf5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.210+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf5ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5636e1ceca30 tx=0x7fb9740093c0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 shutdown_connections 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d07a80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb96a000700 0 client.16056 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1b6de40 0x5636e1cf5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf5ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/3080742556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.211+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 >> 172.21.15.50:0/3080742556 conn(0x5636e1baca80 msgr2=0x5636e1ce0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.212+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 shutdown_connections 2024-05-10T21:13:01.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.212+0000 7fb991d3fb80 1 -- 172.21.15.50:0/3080742556 wait complete. 2024-05-10T21:13:01.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.212+0000 7fb991d3fb80 20 client.16056 trim_cache size 0 max 16384 2024-05-10T21:13:01.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.246+0000 7fb991d3fb80 1 Processor -- start 2024-05-10T21:13:01.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.246+0000 7fb991d3fb80 1 -- start start 2024-05-10T21:13:01.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.246+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.246+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.246+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1c0220 con 0x5636e1b6de40 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1bd9f0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1bb860 con 0x5636e1cebc50 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43026/0 (socket says 172.21.15.50:43026) 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 -- 172.21.15.50:0/2823235878 learned_addr learned my addr 172.21.15.50:0/2823235878 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb981200700 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 -- 172.21.15.50:0/2823235878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 msgr2=0x5636e1cf78e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf78e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 -- 172.21.15.50:0/2823235878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cf1bf0 msgr2=0x5636e1cf0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.247+0000 7fb97be00700 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.248+0000 7fb97be00700 1 -- 172.21.15.50:0/2823235878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e1cdffd0 con 0x5636e1b6de40 2024-05-10T21:13:01.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.248+0000 7fb97be00700 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a75e0 tx=0x7fb96c0aa4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56156c7dd8b7356 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.248+0000 7fb970c00700 1 -- 172.21.15.50:0/2823235878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0aaa70 con 0x5636e1b6de40 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.249+0000 7fb970c00700 1 -- 172.21.15.50:0/2823235878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c0b3140 con 0x5636e1b6de40 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.249+0000 7fb970c00700 1 -- 172.21.15.50:0/2823235878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0b2cb0 con 0x5636e1b6de40 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2823235878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 msgr2=0x5636e1ceb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a75e0 tx=0x7fb96c0aa4f0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2823235878 shutdown_connections 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cf1bf0 0x5636e1cf0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1b6de40 0x5636e1ceb710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2823235878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1cebc50 0x5636e1cf78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2823235878 >> 172.21.15.50:0/2823235878 conn(0x5636e1baca80 msgr2=0x5636e1cf5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2823235878 shutdown_connections 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.250+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2823235878 wait complete. 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.251+0000 7fb991d3fb80 1 -- start start 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.251+0000 7fb991d3fb80 1 Processor -- start 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 0x5636e1cfd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1d0c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1bbeb0 con 0x5636e1cf1bf0 2024-05-10T21:13:01.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1bd320 con 0x5636e1cebc50 2024-05-10T21:13:01.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1bc580 con 0x5636e1b6de40 2024-05-10T21:13:01.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb991d3fb80 10 client.0 _ll_register_callbacks cb 0x5636e1ba1380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb97b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb97b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33262/0 (socket says 172.21.15.50:33262) 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb97b400700 1 -- 172.21.15.50:0/2017625891 learned_addr learned my addr 172.21.15.50:0/2017625891 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.252+0000 7fb97be00700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 0x5636e1cfd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97b400700 1 -- 172.21.15.50:0/2017625891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 msgr2=0x5636e1cfd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97b400700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 0x5636e1cfd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97b400700 1 -- 172.21.15.50:0/2017625891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 msgr2=0x5636e1d0c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb981200700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1d0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97b400700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1d0c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97b400700 1 -- 172.21.15.50:0/2017625891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e1d0c890 con 0x5636e1cebc50 2024-05-10T21:13:01.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.253+0000 7fb97be00700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 0x5636e1cfd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:01.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb97b400700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb964003930 tx=0x7fb964003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240d976de06b766e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:01.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb96a000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:01.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964011cc0 con 0x5636e1cebc50 2024-05-10T21:13:01.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c0b3140 con 0x5636e1cebc50 2024-05-10T21:13:01.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb964004830 con 0x5636e1cebc50 2024-05-10T21:13:01.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb964011e60 con 0x5636e1cebc50 2024-05-10T21:13:01.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.254+0000 7fb991d3fb80 10 client.26204 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:01.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5636e1d6f740 con 0x5636e1cebc50 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c0b3140 con 0x5636e1cebc50 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb96401c4f0 con 0x5636e1cebc50 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb991d3fb80 10 client.26204 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb991d3fb80 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb991d3fb80 20 client.26204 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.255+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d71f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.256+0000 7fb991d3fb80 4 client.26204 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.256+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636e1c154a0 con 0x5636e1d05660 2024-05-10T21:13:01.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.256+0000 7fb97be00700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d71f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:01.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.257+0000 7fb97be00700 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d71f00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d9a905b46c0472 server_cookie=dd3098b84e360235 in_seq=0 out_seq=0 2024-05-10T21:13:01.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.257+0000 7fb96a000700 10 client.26204 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:01.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.257+0000 7fb96a000700 1 -- 172.21.15.50:0/2017625891 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636e1c154a0 con 0x5636e1d05660 2024-05-10T21:13:01.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.257+0000 7fb96a000700 10 client.26204 handle_command_reply: tid=0 2024-05-10T21:13:01.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.257+0000 7fb991d3fb80 1 client.26204 shutdown 2024-05-10T21:13:01.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 msgr2=0x5636e1cfdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb964003930 tx=0x7fb964003cf0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 shutdown_connections 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5636e1d05660 0x5636e1d71f00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e1cf1bf0 0x5636e1d0c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636e1b6de40 0x5636e1cfd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 --2- 172.21.15.50:0/2017625891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1cebc50 0x5636e1cfdc10 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb964003930 tx=0x7fb964003cf0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 >> 172.21.15.50:0/2017625891 conn(0x5636e1baca80 msgr2=0x5636e1cf53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.258+0000 7fb96a000700 0 client.26204 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.259+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 shutdown_connections 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.259+0000 7fb991d3fb80 1 -- 172.21.15.50:0/2017625891 wait complete. 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.259+0000 7fb991d3fb80 20 client.26204 trim_cache size 0 max 16384 2024-05-10T21:13:01.257 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:13:01.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 msgr2=0x7fb95c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 0x7fb95c0b4e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb9740079a0 tx=0x7fb974009920 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 msgr2=0x7fb97c1af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009fd0 tx=0x7fb96c00fb40 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 shutdown_connections 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb95c0b2900 0x7fb95c0b4e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb97c06f710 0x7fb97c1af870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c11d9e0 0x7fb97c1a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 --2- 172.21.15.50:0/1191963641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb97c070060 0x7fb97c1afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 >> 172.21.15.50:0/1191963641 conn(0x7fb97c10ab60 msgr2=0x7fb97c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 shutdown_connections 2024-05-10T21:13:01.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:01.260+0000 7fb983200700 1 -- 172.21.15.50:0/1191963641 wait complete. 2024-05-10T21:13:01.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:00 smithi092 ceph-mon[34415]: pgmap v585: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-05-10T21:13:01.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:00 smithi142 ceph-mon[34846]: pgmap v585: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-05-10T21:13:01.381 INFO:tasks.workunit.client.0.smithi050.stdout: 54 21300 0 13297 0 7306 0 2024-05-10T21:13:03.193 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:13:03.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:13:03.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:02 smithi050 ceph-mon[29546]: pgmap v586: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2024-05-10T21:13:03.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:02 smithi092 ceph-mon[34415]: pgmap v586: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2024-05-10T21:13:03.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:02 smithi142 ceph-mon[34846]: pgmap v586: 97 pgs: 97 active+clean; 230 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2024-05-10T21:13:03.669 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:05.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:04 smithi092 ceph-mon[34415]: pgmap v587: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-10T21:13:05.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:04 smithi142 ceph-mon[34846]: pgmap v587: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-10T21:13:05.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:04 smithi050 ceph-mon[29546]: pgmap v587: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 -- 172.21.15.50:0/2254260927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4109680 msgr2=0x7f3ea4110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 --2- 172.21.15.50:0/2254260927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4109680 0x7f3ea4110fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00b0a0 tx=0x7f3e9c00b3b0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 -- 172.21.15.50:0/2254260927 shutdown_connections 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 --2- 172.21.15.50:0/2254260927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4109680 0x7f3ea4110fe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 --2- 172.21.15.50:0/2254260927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4104400 0x7f3ea4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 --2- 172.21.15.50:0/2254260927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4108d30 0x7f3ea4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 -- 172.21.15.50:0/2254260927 >> 172.21.15.50:0/2254260927 conn(0x7f3ea4079d10 msgr2=0x7f3ea407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 -- 172.21.15.50:0/2254260927 shutdown_connections 2024-05-10T21:13:07.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.342+0000 7f3eaa400700 1 -- 172.21.15.50:0/2254260927 wait complete. 2024-05-10T21:13:07.342 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:06 smithi050 ceph-mon[29546]: pgmap v588: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.344+0000 7f3eaa400700 1 Processor -- start 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.344+0000 7f3eaa400700 1 -- start start 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4104400 0x7f3ea41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 0x7f3ea419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41142b0 con 0x7f3ea4104400 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4114130 con 0x7f3ea4108d30 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3eaa400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4114430 con 0x7f3ea4109680 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4104400 0x7f3ea41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49772/0 (socket says 172.21.15.50:49772) 2024-05-10T21:13:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3ea9a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 0x7f3ea419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.345+0000 7f3ea3e00700 1 -- 172.21.15.50:0/4225870139 learned_addr learned my addr 172.21.15.50:0/4225870139 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 -- 172.21.15.50:0/4225870139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 msgr2=0x7f3ea419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 0x7f3ea419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 -- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4104400 msgr2=0x7f3ea41a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4104400 0x7f3ea41a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 -- 172.21.15.50:0/4225870139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea419b060 con 0x7f3ea4108d30 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 0x7f3ea419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e970 tx=0x7f3e9000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea1600700 1 -- 172.21.15.50:0/4225870139 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004a30 con 0x7f3ea4108d30 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea1600700 1 -- 172.21.15.50:0/4225870139 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9000cc90 con 0x7f3ea4108d30 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea419b350 con 0x7f3ea4108d30 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.346+0000 7f3ea1600700 1 -- 172.21.15.50:0/4225870139 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005610 con 0x7f3ea4108d30 2024-05-10T21:13:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.347+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a7620 con 0x7f3ea4108d30 2024-05-10T21:13:07.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.351+0000 7f3ea1600700 1 -- 172.21.15.50:0/4225870139 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3e9000c7e0 con 0x7f3ea4108d30 2024-05-10T21:13:07.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.351+0000 7f3eba591b80 1 Processor -- start 2024-05-10T21:13:07.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 -- start start 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598ba25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598b9c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598b9d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b598abeaa0 con 0x55b598b9ccc0 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b598a5c7e0 con 0x55b598a1eec0 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3eba591b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b598abaa60 con 0x55b598ba2b00 2024-05-10T21:13:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.352+0000 7f3ea1600700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 0x7f3e8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598b9c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598ba25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea1600700 1 -- 172.21.15.50:0/4225870139 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f3e900e22e0 con 0x7f3ea4108d30 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598b9c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59900/0 (socket says 172.21.15.50:59900) 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea9000700 1 -- 172.21.15.50:0/168258933 learned_addr learned my addr 172.21.15.50:0/168258933 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598b9d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 0x7f3e8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 -- 172.21.15.50:0/168258933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 msgr2=0x55b598b9c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598b9c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 -- 172.21.15.50:0/168258933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 msgr2=0x55b598ba25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598ba25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.353+0000 7f3ea3e00700 1 -- 172.21.15.50:0/168258933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b598a0b950 con 0x55b598b9ccc0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.354+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598b9d0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a95d0 tx=0x7f3e900050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2ec70fff657975 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.354+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 0x7f3e8c0b4e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55b598b9da30 tx=0x7f3e9c00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.354+0000 7f3e96000700 1 -- 172.21.15.50:0/168258933 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900a9220 con 0x55b598b9ccc0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.354+0000 7f3e96000700 1 -- 172.21.15.50:0/168258933 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9000cad0 con 0x55b598b9ccc0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.354+0000 7f3e96000700 1 -- 172.21.15.50:0/168258933 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000fa10 con 0x55b598b9ccc0 2024-05-10T21:13:07.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 -- 172.21.15.50:0/168258933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 msgr2=0x55b598b9d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598b9d0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a95d0 tx=0x7f3e900050b0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 -- 172.21.15.50:0/168258933 shutdown_connections 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598ba25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598b9d0d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 --2- 172.21.15.50:0/168258933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598b9c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 -- 172.21.15.50:0/168258933 >> 172.21.15.50:0/168258933 conn(0x55b598a63ff0 msgr2=0x55b598b97480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.355+0000 7f3eba591b80 1 -- 172.21.15.50:0/168258933 shutdown_connections 2024-05-10T21:13:07.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.356+0000 7f3eba591b80 1 -- 172.21.15.50:0/168258933 wait complete. 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.356+0000 7f3eba591b80 1 -- start start 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 Processor -- start 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b598ba9510 con 0x55b598ba2b00 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b598a24ba0 con 0x55b598b9ccc0 2024-05-10T21:13:07.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b598ba9690 con 0x55b598a1eec0 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.357+0000 7f3eba591b80 10 client.0 _ll_register_callbacks cb 0x55b598a65cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59910/0 (socket says 172.21.15.50:59910) 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49786/0 (socket says 172.21.15.50:49786) 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9000700 1 -- 172.21.15.50:0/1081539124 learned_addr learned my addr 172.21.15.50:0/1081539124 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 -- 172.21.15.50:0/1081539124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 msgr2=0x55b598ba61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 -- 172.21.15.50:0/1081539124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 msgr2=0x55b598ba6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 -- 172.21.15.50:0/1081539124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b598a844a0 con 0x55b598b9ccc0 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea9000700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e900dfdb0 tx=0x7f3e900a5130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e276115961054927 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.358+0000 7f3e8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900c2070 con 0x55b598b9ccc0 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f3e900cc0e0 con 0x55b598b9ccc0 2024-05-10T21:13:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b598b69240 con 0x55b598b9ccc0 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e90088050 con 0x55b598b9ccc0 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3eba591b80 10 client.26216 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b598bb4050 con 0x55b598b9ccc0 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.359+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b598b69240 con 0x55b598b9ccc0 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3e900b22a0 con 0x55b598b9ccc0 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 20 client.26216 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598bb6520 0x55b598bb8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 4 client.26216 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b598acc480 con 0x55b598bb6520 2024-05-10T21:13:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.360+0000 7f3ea9000700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598bb6520 0x55b598bb8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.361+0000 7f3ea9000700 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598bb6520 0x55b598bb8940 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73125fda2a5e014b server_cookie=8b692357197c16b1 in_seq=0 out_seq=0 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.361+0000 7f3e8aa00700 10 client.26216 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3e8aa00700 1 -- 172.21.15.50:0/1081539124 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b598acc480 con 0x55b598bb6520 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3e8aa00700 10 client.26216 handle_command_reply: tid=0 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 client.26216 shutdown 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 msgr2=0x55b598ba66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e900dfdb0 tx=0x7f3e900a5130 comp rx=0 tx=0).stop 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 shutdown_connections 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598bb6520 0x55b598bb8940 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598a1eec0 0x55b598ba61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 --2- 172.21.15.50:0/1081539124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598b9ccc0 0x55b598ba66f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3e8aa00700 0 client.26216 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.362+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 >> 172.21.15.50:0/1081539124 conn(0x55b598a63ff0 msgr2=0x55b598b9b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.363+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 shutdown_connections 2024-05-10T21:13:07.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.363+0000 7f3eba591b80 1 -- 172.21.15.50:0/1081539124 wait complete. 2024-05-10T21:13:07.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.363+0000 7f3eba591b80 20 client.26216 trim_cache size 0 max 16384 2024-05-10T21:13:07.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:06 smithi092 ceph-mon[34415]: pgmap v588: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T21:13:07.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:06 smithi142 ceph-mon[34846]: pgmap v588: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.385+0000 7f3eba591b80 1 Processor -- start 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 -- start start 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 0x55b598ba8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b1290 con 0x55b598ba2b00 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41aea60 con 0x55b598a1eec0 2024-05-10T21:13:07.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3eba591b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ac8d0 con 0x55b598b9ccc0 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3ea9a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.386+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49788/0 (socket says 172.21.15.50:49788) 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 -- 172.21.15.50:0/4224780532 learned_addr learned my addr 172.21.15.50:0/4224780532 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 0x55b598ba8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 -- 172.21.15.50:0/4224780532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 msgr2=0x55b598ba8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 0x55b598ba8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 -- 172.21.15.50:0/4224780532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 msgr2=0x55b598ba6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 -- 172.21.15.50:0/4224780532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b598b93cc0 con 0x55b598a1eec0 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 0x55b598ba8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.387+0000 7f3ea9000700 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b598ba9630 tx=0x7f3e98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f69d876de79dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.388+0000 7f3e95600700 1 -- 172.21.15.50:0/4224780532 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004a30 con 0x55b598a1eec0 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.388+0000 7f3e95600700 1 -- 172.21.15.50:0/4224780532 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9801e8e0 con 0x55b598a1eec0 2024-05-10T21:13:07.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.388+0000 7f3e95600700 1 -- 172.21.15.50:0/4224780532 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98005590 con 0x55b598a1eec0 2024-05-10T21:13:07.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 -- 172.21.15.50:0/4224780532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 msgr2=0x55b598bb2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b598ba9630 tx=0x7f3e98004750 comp rx=0 tx=0).stop 2024-05-10T21:13:07.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 -- 172.21.15.50:0/4224780532 shutdown_connections 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bb2930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598ba2b00 0x55b598ba6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 --2- 172.21.15.50:0/4224780532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598b9ccc0 0x55b598ba8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 -- 172.21.15.50:0/4224780532 >> 172.21.15.50:0/4224780532 conn(0x55b598a63ff0 msgr2=0x55b598b91990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 -- 172.21.15.50:0/4224780532 shutdown_connections 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.389+0000 7f3eba591b80 1 -- 172.21.15.50:0/4224780532 wait complete. 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 -- start start 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 Processor -- start 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598bbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41acf20 con 0x55b598b9ccc0 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ae390 con 0x55b598a1eec0 2024-05-10T21:13:07.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41ad5f0 con 0x55b598ba2b00 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.390+0000 7f3eba591b80 10 client.0 _ll_register_callbacks cb 0x55b598a65cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54094/0 (socket says 172.21.15.50:54094) 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 -- 172.21.15.50:0/2881956552 learned_addr learned my addr 172.21.15.50:0/2881956552 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea9a00700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598bbb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea9000700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 -- 172.21.15.50:0/2881956552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 msgr2=0x55b598bbb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598bbb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 -- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 msgr2=0x55b598bba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.391+0000 7f3ea3e00700 1 -- 172.21.15.50:0/2881956552 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b598c20b00 con 0x55b598b9ccc0 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3ea3e00700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a5040 tx=0x7f3e900b24b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa069ad0a7258604 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:07.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3e8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900c2070 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f3e900cab80 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9801e8e0 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e900a9040 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3eba591b80 10 client.16077 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.392+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b598c21da0 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9801e8e0 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3e900a5240 con 0x55b598b9ccc0 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 20 client.16077 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598c22270 0x55b598c24690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 4 client.16077 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b598ac6520 con 0x55b598c22270 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.393+0000 7f3ea9000700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598c22270 0x55b598c24690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.394+0000 7f3ea9000700 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598c22270 0x55b598c24690 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b790efd83d176a server_cookie=6ee82cc9de0a0a66 in_seq=0 out_seq=0 2024-05-10T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.394+0000 7f3e8aa00700 10 client.16077 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3e8aa00700 1 -- 172.21.15.50:0/2881956552 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b598ac6520 con 0x55b598c22270 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3e8aa00700 10 client.16077 handle_command_reply: tid=0 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 client.16077 shutdown 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 msgr2=0x55b598bbab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a5040 tx=0x7f3e900b24b0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 shutdown_connections 2024-05-10T21:13:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55b598c22270 0x55b598c24690 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b598a1eec0 0x55b598bba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b598ba2b00 0x55b598bbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 --2- 172.21.15.50:0/2881956552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b598b9ccc0 0x55b598bbab20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.395+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 >> 172.21.15.50:0/2881956552 conn(0x55b598a63ff0 msgr2=0x55b598ba8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.396+0000 7f3e8aa00700 0 client.16077 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.396+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 shutdown_connections 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.396+0000 7f3eba591b80 1 -- 172.21.15.50:0/2881956552 wait complete. 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.396+0000 7f3eba591b80 20 client.16077 trim_cache size 0 max 16384 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:13:07.396 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "3bea2b49-3542-4653-acf7-dfd9fed08521", 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 msgr2=0x7f3e8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 0x7f3e8c0b4e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55b598b9da30 tx=0x7f3e9c00b720 comp rx=0 tx=0).stop 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 msgr2=0x7f3ea41a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e970 tx=0x7f3e9000ec80 comp rx=0 tx=0).stop 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 shutdown_connections 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f3e8c0b2900 0x7f3e8c0b4e10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ea4109680 0x7f3ea419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea4104400 0x7f3ea41a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 --2- 172.21.15.50:0/4225870139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4108d30 0x7f3ea41a0cd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.397+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 >> 172.21.15.50:0/4225870139 conn(0x7f3ea4079d10 msgr2=0x7f3ea41013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.398+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 shutdown_connections 2024-05-10T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:07.398+0000 7f3eaa400700 1 -- 172.21.15.50:0/4225870139 wait complete. 2024-05-10T21:13:08.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:08.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:08 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.331 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:13:08.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:13:08.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:08.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:08 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:13:08.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:08 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:09 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:13:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:09 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:09 smithi050 ceph-mon[29546]: pgmap v589: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 KiB/s wr, 107 op/s 2024-05-10T21:13:09.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:09 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:13:09.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:09 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:09.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:09 smithi092 ceph-mon[34415]: pgmap v589: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 KiB/s wr, 107 op/s 2024-05-10T21:13:09.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:09 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:13:09.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:09 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:09.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:09 smithi142 ceph-mon[34846]: pgmap v589: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 KiB/s wr, 107 op/s 2024-05-10T21:13:10.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:10 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 -- 172.21.15.50:0/4144537947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c075040 msgr2=0x7f715c075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 --2- 172.21.15.50:0/4144537947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c075040 0x7f715c075450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7144009860 tx=0x7f714402a340 comp rx=0 tx=0).stop 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 -- 172.21.15.50:0/4144537947 shutdown_connections 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 --2- 172.21.15.50:0/4144537947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c10baa0 0x7f715c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 --2- 172.21.15.50:0/4144537947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075990 0x7f715c10b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 --2- 172.21.15.50:0/4144537947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c075040 0x7f715c075450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 -- 172.21.15.50:0/4144537947 >> 172.21.15.50:0/4144537947 conn(0x7f715c0fecc0 msgr2=0x7f715c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 -- 172.21.15.50:0/4144537947 shutdown_connections 2024-05-10T21:13:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.252+0000 7f7161200640 1 -- 172.21.15.50:0/4144537947 wait complete. 2024-05-10T21:13:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.253+0000 7f7161200640 1 Processor -- start 2024-05-10T21:13:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 -- start start 2024-05-10T21:13:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075040 0x7f715c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 0x7f715c1a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111780 con 0x7f715c075040 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111600 con 0x7f715c075990 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.254+0000 7f7161200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111900 con 0x7f715c10baa0 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49806/0 (socket says 172.21.15.50:49806) 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 -- 172.21.15.50:0/1509233027 learned_addr learned my addr 172.21.15.50:0/1509233027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715b400640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075040 0x7f715c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715be00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 0x7f715c1a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 -- 172.21.15.50:0/1509233027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 msgr2=0x7f715c1a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 0x7f715c1a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 -- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075040 msgr2=0x7f715c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075040 0x7f715c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 -- 172.21.15.50:0/1509233027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1a3ef0 con 0x7f715c075990 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715be00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 0x7f715c1a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.255+0000 7f715aa00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9c0 tx=0x7f714c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.256+0000 7f7153e00640 1 -- 172.21.15.50:0/1509233027 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c00ccc0 con 0x7f715c075990 2024-05-10T21:13:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.256+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1a41e0 con 0x7f715c075990 2024-05-10T21:13:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.256+0000 7f7153e00640 1 -- 172.21.15.50:0/1509233027 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f714c0054d0 con 0x7f715c075990 2024-05-10T21:13:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.256+0000 7f7153e00640 1 -- 172.21.15.50:0/1509233027 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c010430 con 0x7f715c075990 2024-05-10T21:13:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.256+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1a4780 con 0x7f715c075990 2024-05-10T21:13:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.259+0000 7f7169968740 1 Processor -- start 2024-05-10T21:13:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.259+0000 7f7169968740 1 -- start start 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7153e00640 1 -- 172.21.15.50:0/1509233027 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f714c004f80 con 0x7f715c075990 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d204300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55828d168550 con 0x55828ce392f0 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55828d10dbc0 con 0x55828ce414d0 2024-05-10T21:13:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f7169968740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55828d164510 con 0x55828ce621b0 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54118/0 (socket says 172.21.15.50:54118) 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f715b400640 1 -- 172.21.15.50:0/902388077 learned_addr learned my addr 172.21.15.50:0/902388077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f715aa00640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.260+0000 7f715be00640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d204300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 -- 172.21.15.50:0/902388077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 msgr2=0x55828d204300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d204300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 -- 172.21.15.50:0/902388077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 msgr2=0x55828d20b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 -- 172.21.15.50:0/902388077 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55828ce7a210 con 0x55828ce392f0 2024-05-10T21:13:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715be00640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d204300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f715b400640 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f714402a930 tx=0x7f7144005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a52dbefe284d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f7150c00640 1 -- 172.21.15.50:0/902388077 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144041020 con 0x55828ce392f0 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.261+0000 7f7150c00640 1 -- 172.21.15.50:0/902388077 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144030dd0 con 0x55828ce392f0 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7150c00640 1 -- 172.21.15.50:0/902388077 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144037040 con 0x55828ce392f0 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7153e00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 0x7f71400b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7153e00640 1 -- 172.21.15.50:0/1509233027 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f714c0e26e0 con 0x7f715c075990 2024-05-10T21:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f715be00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 0x7f71400b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7169968740 1 -- 172.21.15.50:0/902388077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 msgr2=0x55828d204880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7169968740 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f714402a930 tx=0x7f7144005d30 comp rx=0 tx=0).stop 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7169968740 1 -- 172.21.15.50:0/902388077 shutdown_connections 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.262+0000 7f7169968740 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d204880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 --2- 172.21.15.50:0/902388077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d204300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 -- 172.21.15.50:0/902388077 >> 172.21.15.50:0/902388077 conn(0x55828ce56300 msgr2=0x55828d1ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 -- 172.21.15.50:0/902388077 shutdown_connections 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 -- 172.21.15.50:0/902388077 wait complete. 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f715be00640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 0x7f71400b4cf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55828d2054d0 tx=0x7f7154009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.263+0000 7f7169968740 1 -- start start 2024-05-10T21:13:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 Processor -- start 2024-05-10T21:13:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d215260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55828d20c170 con 0x55828ce392f0 2024-05-10T21:13:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55828d121660 con 0x55828ce414d0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55828d20c2f0 con 0x55828ce621b0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f7169968740 10 client.0 _ll_register_callbacks cb 0x55828d0eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.264+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d215260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d215260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59976/0 (socket says 172.21.15.50:59976) 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715aa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49812/0 (socket says 172.21.15.50:49812) 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715be00640 1 -- 172.21.15.50:0/2336605976 learned_addr learned my addr 172.21.15.50:0/2336605976 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54128/0 (socket says 172.21.15.50:54128) 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 -- 172.21.15.50:0/2336605976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 msgr2=0x55828d215260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d215260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 -- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 msgr2=0x55828d20e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 -- 172.21.15.50:0/2336605976 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55828cf46620 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f715b400640 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7144009780 tx=0x7f71440308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d08b572d997db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.265+0000 7f7149600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144041030 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7144004250 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7144030dd0 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7144057050 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7169968740 10 client.16089 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55828d218190 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.266+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144030dd0 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7144003ce0 con 0x55828ce392f0 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 10 client.16089 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 20 client.16089 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828ce20d80 0x55828d21b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 4 client.16089 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55828d176160 con 0x55828ce20d80 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.267+0000 7f715aa00640 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828ce20d80 0x55828d21b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.268+0000 7f715aa00640 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828ce20d80 0x55828d21b4f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc67526cd50f6f8c server_cookie=ae426a5e5ba6acf in_seq=0 out_seq=0 2024-05-10T21:13:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.268+0000 7f7149600640 10 client.16089 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.268+0000 7f7149600640 1 -- 172.21.15.50:0/2336605976 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55828d176160 con 0x55828ce20d80 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.268+0000 7f7149600640 10 client.16089 handle_command_reply: tid=0 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 client.16089 shutdown 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 msgr2=0x55828d20ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7144009780 tx=0x7f71440308a0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 shutdown_connections 2024-05-10T21:13:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828ce20d80 0x55828d21b4f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce621b0 0x55828d215260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce414d0 0x55828d20e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7149600640 0 client.16089 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 --2- 172.21.15.50:0/2336605976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d20ddc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 >> 172.21.15.50:0/2336605976 conn(0x55828ce56300 msgr2=0x55828d202ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 shutdown_connections 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 1 -- 172.21.15.50:0/2336605976 wait complete. 2024-05-10T21:13:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.269+0000 7f7169968740 20 client.16089 trim_cache size 0 max 16384 2024-05-10T21:13:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 Processor -- start 2024-05-10T21:13:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 -- start start 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d215180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce621b0 0x55828d1fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1afcb0 con 0x55828ce621b0 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1ad4c0 con 0x55828ce392f0 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.287+0000 7f7169968740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1ab310 con 0x55828ce414d0 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49818/0 (socket says 172.21.15.50:49818) 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 -- 172.21.15.50:0/2204738857 learned_addr learned my addr 172.21.15.50:0/2204738857 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715aa00640 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d215180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715be00640 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce621b0 0x55828d1fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 -- 172.21.15.50:0/2204738857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 msgr2=0x55828d215180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d215180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 -- 172.21.15.50:0/2204738857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce621b0 msgr2=0x55828d1fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce621b0 0x55828d1fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 -- 172.21.15.50:0/2204738857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55828d1fb4a0 con 0x55828ce392f0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.288+0000 7f715b400640 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f714402a860 tx=0x7f7144001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9375edcdf8c98a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.289+0000 7f714be00640 1 -- 172.21.15.50:0/2204738857 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144041070 con 0x55828ce392f0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.289+0000 7f714be00640 1 -- 172.21.15.50:0/2204738857 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144004250 con 0x55828ce392f0 2024-05-10T21:13:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.289+0000 7f714be00640 1 -- 172.21.15.50:0/2204738857 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144037040 con 0x55828ce392f0 2024-05-10T21:13:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.289+0000 7f7169968740 1 -- 172.21.15.50:0/2204738857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 msgr2=0x55828d20e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.289+0000 7f7169968740 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f714402a860 tx=0x7f7144001e70 comp rx=0 tx=0).stop 2024-05-10T21:13:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 -- 172.21.15.50:0/2204738857 shutdown_connections 2024-05-10T21:13:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d215180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce621b0 0x55828d1fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 --2- 172.21.15.50:0/2204738857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce392f0 0x55828d20e0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 -- 172.21.15.50:0/2204738857 >> 172.21.15.50:0/2204738857 conn(0x55828ce56300 msgr2=0x55828d20d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 -- 172.21.15.50:0/2204738857 shutdown_connections 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 -- 172.21.15.50:0/2204738857 wait complete. 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 -- start start 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.290+0000 7f7169968740 1 Processor -- start 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d2127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce621b0 0x55828d289fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1ab980 con 0x55828ce392f0 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1acdf0 con 0x55828ce621b0 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1ac050 con 0x55828ce414d0 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f7169968740 10 client.0 _ll_register_callbacks cb 0x7f714c0e8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f715b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54146/0 (socket says 172.21.15.50:54146) 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f715b400640 1 -- 172.21.15.50:0/3999796062 learned_addr learned my addr 172.21.15.50:0/3999796062 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f715be00640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce621b0 0x55828d289fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.291+0000 7f715aa00640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d2127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 -- 172.21.15.50:0/3999796062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 msgr2=0x55828d2127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d2127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 -- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce621b0 msgr2=0x55828d289fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce621b0 0x55828d289fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 -- 172.21.15.50:0/3999796062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55828d28a580 con 0x55828ce392f0 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f715b400640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f714402a930 tx=0x7f7144030990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd8e43cdd810727 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f7149600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144061060 con 0x55828ce392f0 2024-05-10T21:13:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f71440037f0 con 0x55828ce392f0 2024-05-10T21:13:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.292+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7144004250 con 0x55828ce392f0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7144041070 con 0x55828ce392f0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 10 client.16095 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55828d28b810 con 0x55828ce392f0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144004250 con 0x55828ce392f0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7144005260 con 0x55828ce392f0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 10 client.16095 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 20 client.16095 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828cdf61c0 0x55828d28e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 4 client.16095 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.293+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55828d16ffd0 con 0x55828cdf61c0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.294+0000 7f715aa00640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828cdf61c0 0x55828d28e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.294+0000 7f715aa00640 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828cdf61c0 0x55828d28e1d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472d68e34fcdca39 server_cookie=de9cdefc9f7760f8 in_seq=0 out_seq=0 2024-05-10T21:13:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.294+0000 7f7149600640 10 client.16095 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.308+0000 7f7149600640 1 -- 172.21.15.50:0/3999796062 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55828d16ffd0 con 0x55828cdf61c0 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.308+0000 7f7149600640 10 client.16095 handle_command_reply: tid=0 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.308+0000 7f7169968740 1 client.16095 shutdown 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.308+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 msgr2=0x55828d212280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.308+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f714402a930 tx=0x7f7144030990 comp rx=0 tx=0).stop 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 shutdown_connections 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55828cdf61c0 0x55828d28e1d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55828ce414d0 0x55828d2127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55828ce621b0 0x55828d289fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 --2- 172.21.15.50:0/3999796062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55828ce392f0 0x55828d212280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7149600640 0 client.16095 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 >> 172.21.15.50:0/3999796062 conn(0x55828ce56300 msgr2=0x55828d2034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 shutdown_connections 2024-05-10T21:13:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 1 -- 172.21.15.50:0/3999796062 wait complete. 2024-05-10T21:13:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.309+0000 7f7169968740 20 client.16095 trim_cache size 0 max 16384 2024-05-10T21:13:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:13:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:13:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:13:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:13:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 msgr2=0x7f71400b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 0x7f71400b4cf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55828d2054d0 tx=0x7f7154009500 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 msgr2=0x7f715c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9c0 tx=0x7f714c00ee80 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 shutdown_connections 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71400b27e0 0x7f71400b4cf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f715c10baa0 0x7f715c1a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f715c075990 0x7f715c19c8f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 --2- 172.21.15.50:0/1509233027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f715c075040 0x7f715c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 >> 172.21.15.50:0/1509233027 conn(0x7f715c0fecc0 msgr2=0x7f715c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 shutdown_connections 2024-05-10T21:13:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:10.310+0000 7f7161200640 1 -- 172.21.15.50:0/1509233027 wait complete. 2024-05-10T21:13:10.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:10 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:10.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:10 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 -- 172.21.15.50:0/1717369160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810a7c0 msgr2=0x7fdc4810abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 --2- 172.21.15.50:0/1717369160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810a7c0 0x7fdc4810abd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 -- 172.21.15.50:0/1717369160 shutdown_connections 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 --2- 172.21.15.50:0/1717369160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 0x7fdc48117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 --2- 172.21.15.50:0/1717369160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810f6c0 0x7fdc4810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 --2- 172.21.15.50:0/1717369160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810a7c0 0x7fdc4810abd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 -- 172.21.15.50:0/1717369160 >> 172.21.15.50:0/1717369160 conn(0x7fdc48076b00 msgr2=0x7fdc48076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.350 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:11 smithi050 ceph-mon[29546]: pgmap v590: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 124 KiB/s wr, 132 op/s 2024-05-10T21:13:11.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 -- 172.21.15.50:0/1717369160 shutdown_connections 2024-05-10T21:13:11.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.351+0000 7fdc4e000700 1 -- 172.21.15.50:0/1717369160 wait complete. 2024-05-10T21:13:11.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.353+0000 7fdc4e000700 1 Processor -- start 2024-05-10T21:13:11.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.353+0000 7fdc4e000700 1 -- start start 2024-05-10T21:13:11.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 0x7fdc481a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810f6c0 0x7fdc481a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4811a8a0 con 0x7fdc4810a7c0 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4811a720 con 0x7fdc4810b110 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4e000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4811aa20 con 0x7fdc4810f6c0 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54154/0 (socket says 172.21.15.50:54154) 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4cc00700 1 -- 172.21.15.50:0/2058154620 learned_addr learned my addr 172.21.15.50:0/2058154620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc47e00700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 0x7fdc481a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.354+0000 7fdc4d600700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810f6c0 0x7fdc481a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 -- 172.21.15.50:0/2058154620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810f6c0 msgr2=0x7fdc481a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810f6c0 0x7fdc481a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 -- 172.21.15.50:0/2058154620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 msgr2=0x7fdc481a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 0x7fdc481a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 -- 172.21.15.50:0/2058154620 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc481a60a0 con 0x7fdc4810a7c0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.355+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00e9a0 tx=0x7fdc3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.356+0000 7fdc45600700 1 -- 172.21.15.50:0/2058154620 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00ca10 con 0x7fdc4810a7c0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.356+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc481a6390 con 0x7fdc4810a7c0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.356+0000 7fdc45600700 1 -- 172.21.15.50:0/2058154620 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c00cbb0 con 0x7fdc4810a7c0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.356+0000 7fdc45600700 1 -- 172.21.15.50:0/2058154620 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c018680 con 0x7fdc4810a7c0 2024-05-10T21:13:11.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.356+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc481adcb0 con 0x7fdc4810a7c0 2024-05-10T21:13:11.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.361+0000 7fdc45600700 1 -- 172.21.15.50:0/2058154620 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc3c010430 con 0x7fdc4810a7c0 2024-05-10T21:13:11.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.362+0000 7fdc5e18db80 1 Processor -- start 2024-05-10T21:13:11.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.362+0000 7fdc5e18db80 1 -- start start 2024-05-10T21:13:11.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da63ae40 0x5644da7be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5644da6daa20 con 0x5644da7bea80 2024-05-10T21:13:11.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644da678760 con 0x5644da7b8c40 2024-05-10T21:13:11.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644da6d69e0 con 0x5644da63ae40 2024-05-10T21:13:11.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49864/0 (socket says 172.21.15.50:49864) 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54168/0 (socket says 172.21.15.50:54168) 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc47e00700 1 -- 172.21.15.50:0/473964704 learned_addr learned my addr 172.21.15.50:0/473964704 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.363+0000 7fdc4d600700 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da63ae40 0x5644da7be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc45600700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 0x7fdc300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 -- 172.21.15.50:0/473964704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da63ae40 msgr2=0x5644da7be540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da63ae40 0x5644da7be540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 -- 172.21.15.50:0/473964704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 msgr2=0x5644da7b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 -- 172.21.15.50:0/473964704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644da6278d0 con 0x5644da7b8c40 2024-05-10T21:13:11.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc45600700 1 -- 172.21.15.50:0/2058154620 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdc3c0e22c0 con 0x7fdc4810a7c0 2024-05-10T21:13:11.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc4d600700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 0x7fdc300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.364+0000 7fdc47e00700 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc380096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=311a6886b52001c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.365+0000 7fdc36000700 1 -- 172.21.15.50:0/473964704 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38027070 con 0x5644da7b8c40 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.365+0000 7fdc36000700 1 -- 172.21.15.50:0/473964704 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc380056b0 con 0x5644da7b8c40 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.365+0000 7fdc36000700 1 -- 172.21.15.50:0/473964704 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38003ce0 con 0x5644da7b8c40 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 -- 172.21.15.50:0/473964704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 msgr2=0x5644da7b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc4d600700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 0x7fdc300b4e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5644da7b99b0 tx=0x7fdc40009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc380096a0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 -- 172.21.15.50:0/473964704 shutdown_connections 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7b9050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da63ae40 0x5644da7be540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/473964704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 -- 172.21.15.50:0/473964704 >> 172.21.15.50:0/473964704 conn(0x5644da67ff70 msgr2=0x5644da7b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 -- 172.21.15.50:0/473964704 shutdown_connections 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.366+0000 7fdc5e18db80 1 -- 172.21.15.50:0/473964704 wait complete. 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.367+0000 7fdc5e18db80 1 -- start start 2024-05-10T21:13:11.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.367+0000 7fdc5e18db80 1 Processor -- start 2024-05-10T21:13:11.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.367+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da63ae40 0x5644da7c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.367+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5644da7c5490 con 0x5644da7bea80 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644da640b20 con 0x5644da63ae40 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644da7c5610 con 0x5644da7b8c40 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc5e18db80 10 client.0 _ll_register_callbacks cb 0x5644da681c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da63ae40 0x5644da7c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60000/0 (socket says 172.21.15.50:60000) 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc4d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 -- 172.21.15.50:0/3453535086 learned_addr learned my addr 172.21.15.50:0/3453535086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 -- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da63ae40 msgr2=0x5644da7c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da63ae40 0x5644da7c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 -- 172.21.15.50:0/3453535086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 msgr2=0x5644da7c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.368+0000 7fdc47e00700 1 -- 172.21.15.50:0/3453535086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644da6a0420 con 0x5644da7b8c40 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.369+0000 7fdc47e00700 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000c00 tx=0x7fdc38009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43795e221c41e7a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.369+0000 7fdc2aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.369+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38027040 con 0x5644da7b8c40 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.369+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdc38003ca0 con 0x5644da7b8c40 2024-05-10T21:13:11.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.369+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc380056b0 con 0x5644da7b8c40 2024-05-10T21:13:11.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:11 smithi092 ceph-mon[34415]: pgmap v590: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 124 KiB/s wr, 132 op/s 2024-05-10T21:13:11.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc3800b720 con 0x5644da7b8c40 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 10 client.26203 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5644da7c5490 con 0x5644da7b8c40 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc380056b0 con 0x5644da7b8c40 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdc38022830 con 0x5644da7b8c40 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 10 client.26203 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 20 client.26203 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da7d23e0 0x5644da7d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 4 client.26203 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.370+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644da7851c0 con 0x5644da7d23e0 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.371+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da7d23e0 0x5644da7d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.371+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da7d23e0 0x5644da7d4800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99880ead00d92e09 server_cookie=2b7178d69fa3615d in_seq=0 out_seq=0 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.371+0000 7fdc2aa00700 10 client.26203 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:11.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.372+0000 7fdc2aa00700 1 -- 172.21.15.50:0/3453535086 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644da7851c0 con 0x5644da7d23e0 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.372+0000 7fdc2aa00700 10 client.26203 handle_command_reply: tid=0 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.372+0000 7fdc5e18db80 1 client.26203 shutdown 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 msgr2=0x5644da7c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000c00 tx=0x7fdc38009310 comp rx=0 tx=0).stop 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 shutdown_connections 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da7d23e0 0x5644da7d4800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da63ae40 0x5644da7c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da7bea80 0x5644da7c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc2aa00700 0 client.26203 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/3453535086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c26f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 >> 172.21.15.50:0/3453535086 conn(0x5644da67ff70 msgr2=0x5644da7b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 shutdown_connections 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 1 -- 172.21.15.50:0/3453535086 wait complete. 2024-05-10T21:13:11.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.373+0000 7fdc5e18db80 20 client.26203 trim_cache size 0 max 16384 2024-05-10T21:13:11.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:11 smithi142 ceph-mon[34846]: pgmap v590: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 124 KiB/s wr, 132 op/s 2024-05-10T21:13:11.381 INFO:tasks.workunit.client.0.smithi050.stdout: 54 38088 0 22677 0 14066 0 2024-05-10T21:13:11.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 Processor -- start 2024-05-10T21:13:11.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 -- start start 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7bea80 0x5644da7c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b7780 con 0x5644da63ae40 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b4f50 con 0x5644da7b8c40 2024-05-10T21:13:11.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.396+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b2dc0 con 0x5644da7bea80 2024-05-10T21:13:11.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54180/0 (socket says 172.21.15.50:54180) 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 -- 172.21.15.50:0/599570206 learned_addr learned my addr 172.21.15.50:0/599570206 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4d600700 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7bea80 0x5644da7c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 -- 172.21.15.50:0/599570206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7bea80 msgr2=0x5644da7c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7bea80 0x5644da7c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 -- 172.21.15.50:0/599570206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 msgr2=0x5644da7c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.397+0000 7fdc4cc00700 1 -- 172.21.15.50:0/599570206 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644da7afc40 con 0x5644da63ae40 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.398+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0dfd90 tx=0x7fdc3c0b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a6638a731d0c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.398+0000 7fdc35600700 1 -- 172.21.15.50:0/599570206 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0a5770 con 0x5644da63ae40 2024-05-10T21:13:11.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.398+0000 7fdc35600700 1 -- 172.21.15.50:0/599570206 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c0a93c0 con 0x5644da63ae40 2024-05-10T21:13:11.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.398+0000 7fdc35600700 1 -- 172.21.15.50:0/599570206 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0a9020 con 0x5644da63ae40 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 -- 172.21.15.50:0/599570206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 msgr2=0x5644da7b8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0dfd90 tx=0x7fdc3c0b2040 comp rx=0 tx=0).stop 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 -- 172.21.15.50:0/599570206 shutdown_connections 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7bea80 0x5644da7c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7b8700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/599570206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7b8c40 0x5644da7c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 -- 172.21.15.50:0/599570206 >> 172.21.15.50:0/599570206 conn(0x5644da67ff70 msgr2=0x5644da7ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 -- 172.21.15.50:0/599570206 shutdown_connections 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.399+0000 7fdc5e18db80 1 -- 172.21.15.50:0/599570206 wait complete. 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.400+0000 7fdc5e18db80 1 -- start start 2024-05-10T21:13:11.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.400+0000 7fdc5e18db80 1 Processor -- start 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7bea80 0x5644da83b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b3410 con 0x5644da63ae40 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b4880 con 0x5644da7bea80 2024-05-10T21:13:11.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481b3ae0 con 0x5644da7b8c40 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc5e18db80 10 client.0 _ll_register_callbacks cb 0x5644da681c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc47e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc4cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54196/0 (socket says 172.21.15.50:54196) 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc4cc00700 1 -- 172.21.15.50:0/4043120175 learned_addr learned my addr 172.21.15.50:0/4043120175 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.401+0000 7fdc4d600700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7bea80 0x5644da83b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 -- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7bea80 msgr2=0x5644da83b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7bea80 0x5644da83b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 -- 172.21.15.50:0/4043120175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 msgr2=0x5644da7c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 -- 172.21.15.50:0/4043120175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644da7b4d40 con 0x5644da7b8c40 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:11.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc47e00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c70b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc38022c50 tx=0x7fdc380036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8223959753d7d3ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc2aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38027040 con 0x5644da7b8c40 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdc38022480 con 0x5644da7b8c40 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.402+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3c0a93c0 con 0x5644da7b8c40 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.403+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc38048070 con 0x5644da7b8c40 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.403+0000 7fdc5e18db80 10 client.26209 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:11.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.403+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5644da83ceb0 con 0x5644da7b8c40 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.403+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3c0a93c0 con 0x5644da7b8c40 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.403+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdc3803f020 con 0x5644da7b8c40 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 10 client.26209 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 20 client.26209 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da83d590 0x5644da83f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 4 client.26209 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644da6e8400 con 0x5644da83d590 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.404+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da83d590 0x5644da83f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc2aa00700 10 client.26209 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc4cc00700 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da83d590 0x5644da83f9b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160ddc58f7c39500 server_cookie=a3402a2d0623eca7 in_seq=0 out_seq=0 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc2aa00700 1 -- 172.21.15.50:0/4043120175 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5644da6e8400 con 0x5644da83d590 2024-05-10T21:13:11.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc2aa00700 10 client.26209 handle_command_reply: tid=0 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 client.26209 shutdown 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 msgr2=0x5644da7c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c70b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc38022c50 tx=0x7fdc380036f0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 shutdown_connections 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5644da83d590 0x5644da83f9b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644da7bea80 0x5644da83b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644da63ae40 0x5644da7c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 --2- 172.21.15.50:0/4043120175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644da7b8c40 0x5644da7c70b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc2aa00700 0 client.26209 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 >> 172.21.15.50:0/4043120175 conn(0x5644da67ff70 msgr2=0x5644da7c2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 shutdown_connections 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.405+0000 7fdc5e18db80 1 -- 172.21.15.50:0/4043120175 wait complete. 2024-05-10T21:13:11.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.406+0000 7fdc5e18db80 20 client.26209 trim_cache size 0 max 16384 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "3bea2b49-3542-4653-acf7-dfd9fed08521": { 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "3bea2b49-3542-4653-acf7-dfd9fed08521", 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:11.406 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.406+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 msgr2=0x7fdc300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.406+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 0x7fdc300b4e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5644da7b99b0 tx=0x7fdc40009040 comp rx=0 tx=0).stop 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.406+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 msgr2=0x7fdc481a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.406+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00e9a0 tx=0x7fdc3c00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 shutdown_connections 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdc300b2900 0x7fdc300b4e10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4810f6c0 0x7fdc481a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4810a7c0 0x7fdc481a11d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 --2- 172.21.15.50:0/2058154620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc4810b110 0x7fdc481a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:11.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 >> 172.21.15.50:0/2058154620 conn(0x7fdc48076b00 msgr2=0x7fdc48108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:11.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 shutdown_connections 2024-05-10T21:13:11.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:11.407+0000 7fdc4e000700 1 -- 172.21.15.50:0/2058154620 wait complete. 2024-05-10T21:13:12.132 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:12 smithi050 ceph-mon[29546]: pgmap v591: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 103 KiB/s wr, 170 op/s 2024-05-10T21:13:12.211 INFO:tasks.cephfs.filesystem:scrub status for tag:3bea2b49-3542-4653-acf7-dfd9fed08521 - {'path': '/', 'tag': '3bea2b49-3542-4653-acf7-dfd9fed08521', 'options': 'recursive,force'} 2024-05-10T21:13:12.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:12 smithi092 ceph-mon[34415]: pgmap v591: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 103 KiB/s wr, 170 op/s 2024-05-10T21:13:12.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:12 smithi142 ceph-mon[34846]: pgmap v591: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 103 KiB/s wr, 170 op/s 2024-05-10T21:13:15.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:14 smithi092 ceph-mon[34415]: pgmap v592: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 79 KiB/s wr, 203 op/s 2024-05-10T21:13:15.126 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:14 smithi142 ceph-mon[34846]: pgmap v592: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 79 KiB/s wr, 203 op/s 2024-05-10T21:13:15.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:14 smithi050 ceph-mon[29546]: pgmap v592: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 79 KiB/s wr, 203 op/s 2024-05-10T21:13:17.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:16 smithi092 ceph-mon[34415]: pgmap v593: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 20 KiB/s wr, 176 op/s 2024-05-10T21:13:17.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:16 smithi142 ceph-mon[34846]: pgmap v593: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 20 KiB/s wr, 176 op/s 2024-05-10T21:13:17.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:16 smithi050 ceph-mon[29546]: pgmap v593: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 20 KiB/s wr, 176 op/s 2024-05-10T21:13:19.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:18 smithi092 ceph-mon[34415]: pgmap v594: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 23 KiB/s wr, 163 op/s 2024-05-10T21:13:19.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:18 smithi142 ceph-mon[34846]: pgmap v594: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 23 KiB/s wr, 163 op/s 2024-05-10T21:13:19.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:18 smithi050 ceph-mon[29546]: pgmap v594: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 23 KiB/s wr, 163 op/s 2024-05-10T21:13:19.473 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:13:19.981 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:20.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:19 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:13:20.130 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:19 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:13:20.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:19 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:13:21.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:20 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:13:21.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:20 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:13:21.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:20 smithi050 ceph-mon[29546]: pgmap v595: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 30 KiB/s wr, 154 op/s 2024-05-10T21:13:21.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:20 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:13:21.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:20 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:13:21.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:20 smithi092 ceph-mon[34415]: pgmap v595: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 30 KiB/s wr, 154 op/s 2024-05-10T21:13:21.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:20 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:13:21.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:20 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:13:21.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:20 smithi142 ceph-mon[34846]: pgmap v595: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 30 KiB/s wr, 154 op/s 2024-05-10T21:13:21.381 INFO:tasks.workunit.client.0.smithi050.stdout: 54 0 0 0 0 0 0 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 -- 172.21.15.50:0/1035024015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106ec0 msgr2=0x7f7128113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 --2- 172.21.15.50:0/1035024015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106ec0 0x7f7128113350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f712000b0a0 tx=0x7f712000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 -- 172.21.15.50:0/1035024015 shutdown_connections 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 --2- 172.21.15.50:0/1035024015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106ec0 0x7f7128113350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 --2- 172.21.15.50:0/1035024015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f712810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 --2- 172.21.15.50:0/1035024015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106570 0x7f7128106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.321+0000 7f712fc00700 1 -- 172.21.15.50:0/1035024015 >> 172.21.15.50:0/1035024015 conn(0x7f7128102050 msgr2=0x7f71281044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.322+0000 7f712fc00700 1 -- 172.21.15.50:0/1035024015 shutdown_connections 2024-05-10T21:13:22.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.322+0000 7f712fc00700 1 -- 172.21.15.50:0/1035024015 wait complete. 2024-05-10T21:13:22.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 Processor -- start 2024-05-10T21:13:22.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 -- start start 2024-05-10T21:13:22.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:22.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 0x7f712819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:22.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71281166e0 con 0x7f7128106ec0 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7128116560 con 0x7f7128106570 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.323+0000 7f712fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7128116860 con 0x7f712810c5b0 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f7127e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 0x7f712819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57146/0 (socket says 172.21.15.50:57146) 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:40534/0 (socket says 172.21.15.50:40534) 2024-05-10T21:13:22.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712d200700 1 -- 172.21.15.50:0/3244424137 learned_addr learned my addr 172.21.15.50:0/3244424137 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:22.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.324+0000 7f712d200700 1 -- 172.21.15.50:0/3244424137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 msgr2=0x7f71281a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:22.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712d200700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712d200700 1 -- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 msgr2=0x7f712819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:22.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712d200700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 0x7f712819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712d200700 1 -- 172.21.15.50:0/3244424137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a1e80 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712dc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f7127e00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 0x7f712819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712d200700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e970 tx=0x7f711c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c004a30 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c00cc90 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c005610 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.325+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a2170 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.326+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281a9a90 con 0x7f7128106570 2024-05-10T21:13:22.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.327+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71280512c0 con 0x7f7128106570 2024-05-10T21:13:22.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.330+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f711c00c7e0 con 0x7f7128106570 2024-05-10T21:13:22.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.331+0000 7f7125600700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 0x7f71100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:22.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.331+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f711c0e25d0 con 0x7f7128106570 2024-05-10T21:13:22.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.331+0000 7f7127e00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 0x7f71100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:22.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.332+0000 7f7127e00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 0x7f71100b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f71281053d0 tx=0x7f7114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:22.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.333+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f711c054df0 con 0x7f7128106570 2024-05-10T21:13:22.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.623+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f712802e5f0 con 0x7f7128106570 2024-05-10T21:13:22.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.625+0000 7f7125600700 1 -- 172.21.15.50:0/3244424137 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11858 (secure 0 0 0) 0x7f711c0aa2c0 con 0x7f7128106570 2024-05-10T21:13:22.623 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:13:22.623 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:11:36.879959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:13:22.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 msgr2=0x7f71100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:22.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 0x7f71100b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f71281053d0 tx=0x7f7114009260 comp rx=0 tx=0).stop 2024-05-10T21:13:22.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 msgr2=0x7f712819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:22.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e970 tx=0x7f711c00ec80 comp rx=0 tx=0).stop 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 shutdown_connections 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f71100b2880 0x7f71100b4d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7128106570 0x7f712819cfb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f712810c5b0 0x7f71281a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 --2- 172.21.15.50:0/3244424137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7128106ec0 0x7f712819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 >> 172.21.15.50:0/3244424137 conn(0x7f7128102050 msgr2=0x7f71281035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 shutdown_connections 2024-05-10T21:13:22.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:22.628+0000 7f712fc00700 1 -- 172.21.15.50:0/3244424137 wait complete. 2024-05-10T21:13:22.627 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 31 2024-05-10T21:13:23.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:22 smithi050 ceph-mon[29546]: pgmap v596: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 116 op/s 2024-05-10T21:13:23.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:22 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3244424137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:23.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:22 smithi092 ceph-mon[34415]: pgmap v596: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 116 op/s 2024-05-10T21:13:23.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:22 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3244424137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:23.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:22 smithi142 ceph-mon[34846]: pgmap v596: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 116 op/s 2024-05-10T21:13:23.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:22 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3244424137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.536+0000 7f9b02800640 1 -- 172.21.15.50:0/2028274178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc1143d0 msgr2=0x7f9afc1167f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.536+0000 7f9b02800640 1 --2- 172.21.15.50:0/2028274178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc1143d0 0x7f9afc1167f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009be0 tx=0x7f9af402a6c0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 -- 172.21.15.50:0/2028274178 shutdown_connections 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 --2- 172.21.15.50:0/2028274178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc1143d0 0x7f9afc1167f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 --2- 172.21.15.50:0/2028274178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc06d4a0 0x7f9afc113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 --2- 172.21.15.50:0/2028274178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06cb50 0x7f9afc06cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 -- 172.21.15.50:0/2028274178 >> 172.21.15.50:0/2028274178 conn(0x7f9afc072870 msgr2=0x7f9afc109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 -- 172.21.15.50:0/2028274178 shutdown_connections 2024-05-10T21:13:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.537+0000 7f9b02800640 1 -- 172.21.15.50:0/2028274178 wait complete. 2024-05-10T21:13:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.538+0000 7f9b02800640 1 Processor -- start 2024-05-10T21:13:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.538+0000 7f9b02800640 1 -- start start 2024-05-10T21:13:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 0x7f9afc1a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 0x7f9afc1ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc11a170 con 0x7f9afc1143d0 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc119ff0 con 0x7f9afc06cb50 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9b02800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc11a2f0 con 0x7f9afc06d4a0 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9afb400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9afb400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57158/0 (socket says 172.21.15.50:57158) 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 0x7f9afc1a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9afb400640 1 -- 172.21.15.50:0/3174665714 learned_addr learned my addr 172.21.15.50:0/3174665714 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.539+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 0x7f9afc1ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 -- 172.21.15.50:0/3174665714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 msgr2=0x7f9afc1a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 -- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 msgr2=0x7f9afc1ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 0x7f9afc1ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 -- 172.21.15.50:0/3174665714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc1ac8c0 con 0x7f9afc06d4a0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9afb400640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 0x7f9afc1ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 0x7f9afc1a52d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9afc06eba0 tx=0x7f9ae400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af8c00640 1 -- 172.21.15.50:0/3174665714 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400ccd0 con 0x7f9afc06d4a0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af8c00640 1 -- 172.21.15.50:0/3174665714 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae40035f0 con 0x7f9afc06d4a0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc1acbb0 con 0x7f9afc06d4a0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.540+0000 7f9af8c00640 1 -- 172.21.15.50:0/3174665714 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4010690 con 0x7f9afc06d4a0 2024-05-10T21:13:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.541+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1ad100 con 0x7f9afc06d4a0 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 Processor -- start 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 -- start start 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cb7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9c14550 con 0x55a9f98e52f0 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9bb9bc0 con 0x55a9f990e1b0 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.544+0000 7f9b097b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9c10510 con 0x55a9f98ed4d0 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36970/0 (socket says 172.21.15.50:36970) 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 -- 172.21.15.50:0/3029714134 learned_addr learned my addr 172.21.15.50:0/3029714134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9af8c00640 1 -- 172.21.15.50:0/3174665714 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ae4010830 con 0x7f9afc06d4a0 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 -- 172.21.15.50:0/3029714134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 msgr2=0x55a9f9cb7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cb7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 -- 172.21.15.50:0/3029714134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 msgr2=0x55a9f9cb0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cb0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afb400640 1 -- 172.21.15.50:0/3029714134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9f9926210 con 0x55a9f98e52f0 2024-05-10T21:13:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.545+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.546+0000 7f9afb400640 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0115f0 tx=0x7f9aec011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a27076be83cf82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.546+0000 7f9af0c00640 1 -- 172.21.15.50:0/3029714134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004030 con 0x55a9f98e52f0 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.546+0000 7f9af0c00640 1 -- 172.21.15.50:0/3029714134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec0054d0 con 0x55a9f98e52f0 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.546+0000 7f9af0c00640 1 -- 172.21.15.50:0/3029714134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004b80 con 0x55a9f98e52f0 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.547+0000 7f9af8c00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 0x7f9ae00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.547+0000 7f9af8c00640 1 -- 172.21.15.50:0/3174665714 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9ae40e2330 con 0x7f9afc06d4a0 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.547+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 0x7f9ae00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.547+0000 7f9b097b9740 1 -- 172.21.15.50:0/3029714134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 msgr2=0x55a9f9cb0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.547+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0115f0 tx=0x7f9aec011ab0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 -- 172.21.15.50:0/3029714134 shutdown_connections 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cb7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb0680 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3029714134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 -- 172.21.15.50:0/3029714134 >> 172.21.15.50:0/3029714134 conn(0x55a9f9902300 msgr2=0x55a9f9caba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 -- 172.21.15.50:0/3029714134 shutdown_connections 2024-05-10T21:13:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9b097b9740 1 -- 172.21.15.50:0/3029714134 wait complete. 2024-05-10T21:13:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.548+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 0x7f9ae00b4cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9af402abe0 tx=0x7f9af4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.549+0000 7f9b097b9740 1 -- start start 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.549+0000 7f9b097b9740 1 Processor -- start 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.549+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.549+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.549+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9b097b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9cb7fd0 con 0x55a9f98ed4d0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9b097b9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9bcd660 con 0x55a9f990e1b0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9b097b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a9f9cb8150 con 0x55a9f98e52f0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9b097b9740 10 client.0 _ll_register_callbacks cb 0x55a9f9b97400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9afb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36982/0 (socket says 172.21.15.50:36982) 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 -- 172.21.15.50:0/2294065581 learned_addr learned my addr 172.21.15.50:0/2294065581 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9afbe00640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 -- 172.21.15.50:0/2294065581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 msgr2=0x55a9f9cb9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 -- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 msgr2=0x55a9f9cc1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cc1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.550+0000 7f9af2a00640 1 -- 172.21.15.50:0/2294065581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9f99f2620 con 0x55a9f98ed4d0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9af2a00640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40a9680 tx=0x7f9ae4005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eae9630156dfca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9ae9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4009510 con 0x55a9f98ed4d0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9ae4003a80 con 0x55a9f98ed4d0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec0054d0 con 0x55a9f98ed4d0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9ae40a9980 con 0x55a9f98ed4d0 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9b097b9740 10 client.16131 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.551+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9f9cc4020 con 0x55a9f98ed4d0 2024-05-10T21:13:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec0054d0 con 0x55a9f98ed4d0 2024-05-10T21:13:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae40b2350 con 0x55a9f98ed4d0 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 10 client.16131 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 20 client.16131 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98ccd80 0x55a9f9cc7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 4 client.16131 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9f9c22160 con 0x55a9f98ccd80 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.552+0000 7f9afb400640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98ccd80 0x55a9f9cc7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.553+0000 7f9afb400640 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98ccd80 0x55a9f9cc7740 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfae5ce041f9965 server_cookie=821e11597f942162 in_seq=0 out_seq=0 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.553+0000 7f9ae9600640 10 client.16131 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9ae9600640 1 -- 172.21.15.50:0/2294065581 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a9f9c22160 con 0x55a9f98ccd80 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9ae9600640 10 client.16131 handle_command_reply: tid=0 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 client.16131 shutdown 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 msgr2=0x55a9f9cba190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40a9680 tx=0x7f9ae4005790 comp rx=0 tx=0).stop 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 shutdown_connections 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98ccd80 0x55a9f9cc7740 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cc1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cba190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2294065581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cb9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9ae9600640 0 client.16131 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.554+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 >> 172.21.15.50:0/2294065581 conn(0x55a9f9902300 msgr2=0x55a9f9cae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.555+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 shutdown_connections 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.555+0000 7f9b097b9740 1 -- 172.21.15.50:0/2294065581 wait complete. 2024-05-10T21:13:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.555+0000 7f9b097b9740 20 client.16131 trim_cache size 0 max 16384 2024-05-10T21:13:23.564 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id 67c3b3df --release --await' 2024-05-10T21:13:23.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db --set-id 67c3b3df --release --await 2024-05-10T21:13:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 Processor -- start 2024-05-10T21:13:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 -- start start 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b8520 con 0x55a9f990e1b0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b5d30 con 0x55a9f98ed4d0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9b097b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b3b80 con 0x55a9f98e52f0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36992/0 (socket says 172.21.15.50:36992) 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.575+0000 7f9afbe00640 1 -- 172.21.15.50:0/2531461973 learned_addr learned my addr 172.21.15.50:0/2531461973 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9af2a00640 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afb400640 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 -- 172.21.15.50:0/2531461973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 msgr2=0x55a9f9cabb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cabb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 -- 172.21.15.50:0/2531461973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f98ed4d0 msgr2=0x55a9f9cc0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cc0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 -- 172.21.15.50:0/2531461973 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9f9ca70b0 con 0x55a9f990e1b0 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.576+0000 7f9afbe00640 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009be0 tx=0x7f9af4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e28a9fd86eb5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.577+0000 7f9aebe00640 1 -- 172.21.15.50:0/2531461973 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4036040 con 0x55a9f990e1b0 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.577+0000 7f9aebe00640 1 -- 172.21.15.50:0/2531461973 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4050070 con 0x55a9f990e1b0 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.577+0000 7f9aebe00640 1 -- 172.21.15.50:0/2531461973 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4055d30 con 0x55a9f990e1b0 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 -- 172.21.15.50:0/2531461973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 msgr2=0x55a9f9caa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009be0 tx=0x7f9af4040000 comp rx=0 tx=0).stop 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 -- 172.21.15.50:0/2531461973 shutdown_connections 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f990e1b0 0x55a9f9caa000 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 --2- 172.21.15.50:0/2531461973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98e52f0 0x55a9f9cabb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 -- 172.21.15.50:0/2531461973 >> 172.21.15.50:0/2531461973 conn(0x55a9f9902300 msgr2=0x55a9f9cab450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 -- 172.21.15.50:0/2531461973 shutdown_connections 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.578+0000 7f9b097b9740 1 -- 172.21.15.50:0/2531461973 wait complete. 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 -- start start 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 Processor -- start 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cbe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b41f0 con 0x55a9f98e52f0 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b5660 con 0x55a9f990e1b0 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1b48c0 con 0x55a9f98ed4d0 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.579+0000 7f9b097b9740 10 client.0 _ll_register_callbacks cb 0x7f9ae4004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36996/0 (socket says 172.21.15.50:36996) 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 -- 172.21.15.50:0/3148905833 learned_addr learned my addr 172.21.15.50:0/3148905833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cbe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 -- 172.21.15.50:0/3148905833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 msgr2=0x55a9f9cbe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cbe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 -- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 msgr2=0x55a9f9cbf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cbf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afb400640 1 -- 172.21.15.50:0/3148905833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9f9cbfd00 con 0x55a9f98e52f0 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cbe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.580+0000 7f9afbe00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9afb400640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0097f0 tx=0x7f9aec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc49365e841333d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9ae9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec032070 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9aec005310 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af4050070 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9aec03d050 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9b097b9740 10 client.16137 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.581+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9f9cc1000 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4050070 con 0x55a9f98e52f0 2024-05-10T21:13:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9aec02bce0 con 0x55a9f98e52f0 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 10 client.16137 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 20 client.16137 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98a21c0 0x55a9f9d31e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 4 client.16137 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9f9c1bfd0 con 0x55a9f98a21c0 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.582+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98a21c0 0x55a9f9d31e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.583+0000 7f9af2a00640 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98a21c0 0x55a9f9d31e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce416c2d9947f1b5 server_cookie=eaf37786008de1df in_seq=0 out_seq=0 2024-05-10T21:13:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.583+0000 7f9ae9600640 10 client.16137 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9ae9600640 1 -- 172.21.15.50:0/3148905833 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9f9c1bfd0 con 0x55a9f98a21c0 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9ae9600640 10 client.16137 handle_command_reply: tid=0 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 client.16137 shutdown 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 msgr2=0x55a9f9cbe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0097f0 tx=0x7f9aec005c50 comp rx=0 tx=0).stop 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 shutdown_connections 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55a9f98a21c0 0x55a9f9d31e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f98ed4d0 0x55a9f9cbe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9f990e1b0 0x55a9f9cbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9ae9600640 0 client.16137 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 --2- 172.21.15.50:0/3148905833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a9f98e52f0 0x55a9f9cbe320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.597+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 >> 172.21.15.50:0/3148905833 conn(0x55a9f9902300 msgr2=0x55a9f9cae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.598+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 shutdown_connections 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.598+0000 7f9b097b9740 1 -- 172.21.15.50:0/3148905833 wait complete. 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.598+0000 7f9b097b9740 20 client.16137 trim_cache size 0 max 16384 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 msgr2=0x7f9ae00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 0x7f9ae00b4cf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9af402abe0 tx=0x7f9af4005fd0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 msgr2=0x7f9afc1a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 0x7f9afc1a52d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9afc06eba0 tx=0x7f9ae400ee50 comp rx=0 tx=0).stop 2024-05-10T21:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 shutdown_connections 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9ae00b27e0 0x7f9ae00b4cf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06d4a0 0x7f9afc1a52d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc1143d0 0x7f9afc1ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 --2- 172.21.15.50:0/3174665714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9afc06cb50 0x7f9afc1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 >> 172.21.15.50:0/3174665714 conn(0x7f9afc072870 msgr2=0x7f9afc10fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 shutdown_connections 2024-05-10T21:13:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:23.599+0000 7f9b02800640 1 -- 172.21.15.50:0/3174665714 wait complete. 2024-05-10T21:13:24.012 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:25.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:24 smithi050 ceph-mon[29546]: pgmap v597: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 KiB/s wr, 62 op/s 2024-05-10T21:13:25.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:24 smithi092 ceph-mon[34415]: pgmap v597: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 KiB/s wr, 62 op/s 2024-05-10T21:13:25.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:24 smithi142 ceph-mon[34846]: pgmap v597: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 KiB/s wr, 62 op/s 2024-05-10T21:13:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 -- 172.21.15.50:0/3172561091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 msgr2=0x7fd4b8109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3172561091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b8109150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80099c0 tx=0x7fd4a8009cd0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 -- 172.21.15.50:0/3172561091 shutdown_connections 2024-05-10T21:13:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3172561091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8104410 0x7fd4b8104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3172561091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8109690 0x7fd4b8110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3172561091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b8109150 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 -- 172.21.15.50:0/3172561091 >> 172.21.15.50:0/3172561091 conn(0x7fd4b8079d50 msgr2=0x7fd4b807a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 -- 172.21.15.50:0/3172561091 shutdown_connections 2024-05-10T21:13:26.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.314+0000 7fd4bf200700 1 -- 172.21.15.50:0/3172561091 wait complete. 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.315+0000 7fd4bf200700 1 Processor -- start 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 -- start start 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 0x7fd4b81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81142a0 con 0x7fd4b8108d40 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8114120 con 0x7fd4b8109690 2024-05-10T21:13:26.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.316+0000 7fd4bf200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8114420 con 0x7fd4b8104410 2024-05-10T21:13:26.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4be800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 0x7fd4b81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52178/0 (socket says 172.21.15.50:52178) 2024-05-10T21:13:26.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 -- 172.21.15.50:0/3801060282 learned_addr learned my addr 172.21.15.50:0/3801060282 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:26.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bd400700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 -- 172.21.15.50:0/3801060282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 msgr2=0x7fd4b81a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 0x7fd4b81a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 -- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 msgr2=0x7fd4b819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 -- 172.21.15.50:0/3801060282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b81a1d40 con 0x7fd4b8104410 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4be800700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 0x7fd4b81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.317+0000 7fd4bde00700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e9a0 tx=0x7fd4ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.318+0000 7fd4b2a00700 1 -- 172.21.15.50:0/3801060282 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00ca10 con 0x7fd4b8104410 2024-05-10T21:13:26.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.318+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81a2030 con 0x7fd4b8104410 2024-05-10T21:13:26.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.318+0000 7fd4b2a00700 1 -- 172.21.15.50:0/3801060282 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ac00cbb0 con 0x7fd4b8104410 2024-05-10T21:13:26.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.318+0000 7fd4b2a00700 1 -- 172.21.15.50:0/3801060282 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac018680 con 0x7fd4b8104410 2024-05-10T21:13:26.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.318+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a9950 con 0x7fd4b8104410 2024-05-10T21:13:26.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.322+0000 7fd4b2a00700 1 -- 172.21.15.50:0/3801060282 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4ac010430 con 0x7fd4b8104410 2024-05-10T21:13:26.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.323+0000 7fd4cf2beb80 1 Processor -- start 2024-05-10T21:13:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.323+0000 7fd4cf2beb80 1 -- start start 2024-05-10T21:13:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b8b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 0x5586f1b8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b90220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5586f1aacb20 con 0x5586f1b8fe10 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5586f1a4a860 con 0x5586f1b8b610 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.324+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5586f1aa8ae0 con 0x5586f1a0cf40 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4be800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b8b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 0x5586f1b8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4b2a00700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 0x7fd4a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b90220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4be800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b8b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52186/0 (socket says 172.21.15.50:52186) 2024-05-10T21:13:26.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4be800700 1 -- 172.21.15.50:0/525928593 learned_addr learned my addr 172.21.15.50:0/525928593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4b2a00700 1 -- 172.21.15.50:0/3801060282 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fd4ac0e2330 con 0x7fd4b8104410 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 -- 172.21.15.50:0/525928593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 msgr2=0x5586f1b8b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b8b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 -- 172.21.15.50:0/525928593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 msgr2=0x5586f1b8f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 0x5586f1b8f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bd400700 1 -- 172.21.15.50:0/525928593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f19f99d0 con 0x5586f1b8fe10 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4bde00700 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 0x5586f1b8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.325+0000 7fd4be800700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 0x7fd4a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.326+0000 7fd4bd400700 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b90220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b580 tx=0x7fd4a8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90519b06f9edc49a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.326+0000 7fd4a7400700 1 -- 172.21.15.50:0/525928593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8027070 con 0x5586f1b8fe10 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.326+0000 7fd4a7400700 1 -- 172.21.15.50:0/525928593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a80037c0 con 0x5586f1b8fe10 2024-05-10T21:13:26.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.326+0000 7fd4a7400700 1 -- 172.21.15.50:0/525928593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800b990 con 0x5586f1b8fe10 2024-05-10T21:13:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/525928593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 msgr2=0x5586f1b90220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b90220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b580 tx=0x7fd4a8005040 comp rx=0 tx=0).stop 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4be800700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 0x7fd4a00b4e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5586f1b8bf80 tx=0x7fd4b4004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/525928593 shutdown_connections 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b90220 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b8b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/525928593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8b610 0x5586f1b8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/525928593 >> 172.21.15.50:0/525928593 conn(0x5586f1a4bb80 msgr2=0x5586f1b85df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/525928593 shutdown_connections 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.327+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/525928593 wait complete. 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.328+0000 7fd4cf2beb80 1 -- start start 2024-05-10T21:13:26.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.328+0000 7fd4cf2beb80 1 Processor -- start 2024-05-10T21:13:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b94ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8b610 0x5586f1b95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5586f1b97d60 con 0x5586f1b8fe10 2024-05-10T21:13:26.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5586f1a12c20 con 0x5586f1a0cf40 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5586f1b97ee0 con 0x5586f1b8b610 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4cf2beb80 10 client.0 _ll_register_callbacks cb 0x5586f1a40480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4be800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8b610 0x5586f1b95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b94ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4be800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b95540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43804/0 (socket says 172.21.15.50:43804) 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.329+0000 7fd4be800700 1 -- 172.21.15.50:0/1380198550 learned_addr learned my addr 172.21.15.50:0/1380198550 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 -- 172.21.15.50:0/1380198550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8b610 msgr2=0x5586f1b95000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8b610 0x5586f1b95000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 -- 172.21.15.50:0/1380198550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 msgr2=0x5586f1b95540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b95540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 -- 172.21.15.50:0/1380198550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f1a72520 con 0x5586f1a0cf40 2024-05-10T21:13:26.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4bde00700 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b94ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0a6b90 tx=0x7fd4ac010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3a18195920c3ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd49be00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0aa020 con 0x5586f1a0cf40 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fd4ac0cc0e0 con 0x5586f1a0cf40 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.330+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a80037c0 con 0x5586f1a0cf40 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4ac0a9920 con 0x5586f1a0cf40 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 10 client.26294 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5586f1ba2950 con 0x5586f1a0cf40 2024-05-10T21:13:26.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a80037c0 con 0x5586f1a0cf40 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd4ac0a8b80 con 0x5586f1a0cf40 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 10 client.26294 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 10 client.26294 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 20 client.26294 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1ba4fe0 0x5586f1ba74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 4 client.26294 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.331+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f1b57170 con 0x5586f1ba4fe0 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.332+0000 7fd4bd400700 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1ba4fe0 0x5586f1ba74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.332+0000 7fd4bd400700 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1ba4fe0 0x5586f1ba74b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae545b618f5fb47 server_cookie=b2f24e3e38175e50 in_seq=0 out_seq=0 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.332+0000 7fd49be00700 10 client.26294 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.333+0000 7fd49be00700 1 -- 172.21.15.50:0/1380198550 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586f1b57170 con 0x5586f1ba4fe0 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.333+0000 7fd49be00700 10 client.26294 handle_command_reply: tid=0 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.333+0000 7fd4cf2beb80 1 client.26294 shutdown 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 msgr2=0x5586f1b94ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b94ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0a6b90 tx=0x7fd4ac010c10 comp rx=0 tx=0).stop 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 shutdown_connections 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1ba4fe0 0x5586f1ba74b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b94ac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8fe10 0x5586f1b95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd49be00700 0 client.26294 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1380198550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8b610 0x5586f1b95000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 >> 172.21.15.50:0/1380198550 conn(0x5586f1a4bb80 msgr2=0x5586f1b85a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 shutdown_connections 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1380198550 wait complete. 2024-05-10T21:13:26.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.334+0000 7fd4cf2beb80 20 client.26294 trim_cache size 0 max 16384 2024-05-10T21:13:26.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.356+0000 7fd4cf2beb80 1 Processor -- start 2024-05-10T21:13:26.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.356+0000 7fd4cf2beb80 1 -- start start 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b96ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8fe10 0x5586f1bacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81b3390 con 0x5586f1b8b610 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81b0b60 con 0x5586f1a0cf40 2024-05-10T21:13:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81aea30 con 0x5586f1b8fe10 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b96ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43810/0 (socket says 172.21.15.50:43810) 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4bd400700 1 -- 172.21.15.50:0/1191116923 learned_addr learned my addr 172.21.15.50:0/1191116923 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.357+0000 7fd4be800700 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8fe10 0x5586f1bacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 -- 172.21.15.50:0/1191116923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8fe10 msgr2=0x5586f1bacc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8fe10 0x5586f1bacc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 -- 172.21.15.50:0/1191116923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 msgr2=0x5586f1b96ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b96ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 -- 172.21.15.50:0/1191116923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f1b80b30 con 0x5586f1b8b610 2024-05-10T21:13:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bde00700 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b96ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4bd400700 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005b40 tx=0x7fd4a8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d3059a80e118f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.358+0000 7fd4a6a00700 1 -- 172.21.15.50:0/1191116923 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8027040 con 0x5586f1b8b610 2024-05-10T21:13:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4a6a00700 1 -- 172.21.15.50:0/1191116923 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a800b790 con 0x5586f1b8b610 2024-05-10T21:13:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4a6a00700 1 -- 172.21.15.50:0/1191116923 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80303d0 con 0x5586f1b8b610 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1191116923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 msgr2=0x5586f1b95380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005b40 tx=0x7fd4a8005960 comp rx=0 tx=0).stop 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1191116923 shutdown_connections 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1a0cf40 0x5586f1b96ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1b8fe10 0x5586f1bacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/1191116923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b95380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.359+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1191116923 >> 172.21.15.50:0/1191116923 conn(0x5586f1a4bb80 msgr2=0x5586f1b8a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.360+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1191116923 shutdown_connections 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.360+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/1191116923 wait complete. 2024-05-10T21:13:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.360+0000 7fd4cf2beb80 1 -- start start 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.360+0000 7fd4cf2beb80 1 Processor -- start 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8fe10 0x5586f1b9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81af080 con 0x5586f1b8b610 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81b0490 con 0x5586f1b8fe10 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81af6f0 con 0x5586f1a0cf40 2024-05-10T21:13:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4cf2beb80 10 client.0 _ll_register_callbacks cb 0x5586f1a40480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4bd400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43824/0 (socket says 172.21.15.50:43824) 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4bd400700 1 -- 172.21.15.50:0/2236157012 learned_addr learned my addr 172.21.15.50:0/2236157012 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4bde00700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.361+0000 7fd4be800700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8fe10 0x5586f1b9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 -- 172.21.15.50:0/2236157012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 msgr2=0x5586f1b9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 -- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8fe10 msgr2=0x5586f1b9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8fe10 0x5586f1b9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 -- 172.21.15.50:0/2236157012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f1b9ada0 con 0x5586f1b8b610 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4bd400700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b620 tx=0x7fd4a800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fcc67b9ca49938 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd49be00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8027040 con 0x5586f1b8b610 2024-05-10T21:13:26.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a800b790 con 0x5586f1b8b610 2024-05-10T21:13:26.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.362+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fd4a8005040 con 0x5586f1b8b610 2024-05-10T21:13:26.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4a8005240 con 0x5586f1b8b610 2024-05-10T21:13:26.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 10 client.16155 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:26.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5586f1c10740 con 0x5586f1b8b610 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a800b790 con 0x5586f1b8b610 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd4a8021030 con 0x5586f1b8b610 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 10 client.16155 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 20 client.16155 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1c10910 0x5586f1c12d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 4 client.16155 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "67c3b3df", "release": true, "await": true}] 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.363+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "set_id": "67c3b3df", "release": true, "await": true}) -- 0x5586f1aba500 con 0x5586f1c10910 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.364+0000 7fd4bde00700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1c10910 0x5586f1c12d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.364+0000 7fd4bde00700 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1c10910 0x5586f1c12d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db043ac2f94e3683 server_cookie=24b907214c229b81 in_seq=0 out_seq=0 2024-05-10T21:13:26.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.364+0000 7fd49be00700 10 client.16155 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.490+0000 7fd49be00700 1 -- 172.21.15.50:0/2236157012 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5586f1aba500 con 0x5586f1c10910 2024-05-10T21:13:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.490+0000 7fd49be00700 10 client.16155 handle_command_reply: tid=0 2024-05-10T21:13:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.490+0000 7fd4cf2beb80 1 client.16155 shutdown 2024-05-10T21:13:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 msgr2=0x5586f1b9a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b620 tx=0x7fd4a800b650 comp rx=0 tx=0).stop 2024-05-10T21:13:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 shutdown_connections 2024-05-10T21:13:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5586f1c10910 0x5586f1c12d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5586f1b8fe10 0x5586f1b9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5586f1a0cf40 0x5586f1b9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 --2- 172.21.15.50:0/2236157012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5586f1b8b610 0x5586f1b9a140 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 >> 172.21.15.50:0/2236157012 conn(0x5586f1a4bb80 msgr2=0x5586f1b96980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd49be00700 0 client.16155 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.491+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 shutdown_connections 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.492+0000 7fd4cf2beb80 1 -- 172.21.15.50:0/2236157012 wait complete. 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.492+0000 7fd4cf2beb80 20 client.16155 trim_cache size 0 max 16384 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 31, 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:13:26.490 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 12, 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "67c3b3df": { 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 12, 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:13:26.491 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 85.2, 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RELEASED", 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:26.492 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.493+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 msgr2=0x7fd4a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.493+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 0x7fd4a00b4e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5586f1b8bf80 tx=0x7fd4b4004ad0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.493+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 msgr2=0x7fd4b819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.493+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e9a0 tx=0x7fd4ac00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 shutdown_connections 2024-05-10T21:13:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4a00b2900 0x7fd4a00b4e10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4b8109690 0x7fd4b81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4b8104410 0x7fd4b819ce70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 --2- 172.21.15.50:0/3801060282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b8108d40 0x7fd4b819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 >> 172.21.15.50:0/3801060282 conn(0x7fd4b8079d50 msgr2=0x7fd4b8110d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 shutdown_connections 2024-05-10T21:13:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:26.494+0000 7fd4bf200700 1 -- 172.21.15.50:0/3801060282 wait complete. 2024-05-10T21:13:27.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:26 smithi050 ceph-mon[29546]: pgmap v598: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-10T21:13:27.168 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:26 smithi092 ceph-mon[34415]: pgmap v598: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-10T21:13:27.375 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '67c3b3df', seconds elapsed: 61.8 2024-05-10T21:13:27.375 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.15 2024-05-10T21:13:27.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:13:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:26 smithi142 ceph-mon[34846]: pgmap v598: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-10T21:13:27.975 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:29.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:28 smithi050 ceph-mon[29546]: pgmap v599: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-05-10T21:13:29.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:28 smithi092 ceph-mon[34415]: pgmap v599: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-05-10T21:13:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:28 smithi142 ceph-mon[34846]: pgmap v599: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-05-10T21:13:30.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.528+0000 7f0058200700 1 -- 172.21.15.50:0/3783566945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00501034c0 msgr2=0x7f00501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:30.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.528+0000 7f0058200700 1 --2- 172.21.15.50:0/3783566945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00501034c0 0x7f00501038d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f00440099c0 tx=0x7f0044009cd0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.529+0000 7f0058200700 1 -- 172.21.15.50:0/3783566945 shutdown_connections 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.529+0000 7f0058200700 1 --2- 172.21.15.50:0/3783566945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0050108740 0x7f00501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.529+0000 7f0058200700 1 --2- 172.21.15.50:0/3783566945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00501034c0 0x7f00501038d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.529+0000 7f0058200700 1 --2- 172.21.15.50:0/3783566945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050107df0 0x7f0050108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.529+0000 7f0058200700 1 -- 172.21.15.50:0/3783566945 >> 172.21.15.50:0/3783566945 conn(0x7f00500fedc0 msgr2=0x7f00501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.530+0000 7f0058200700 1 -- 172.21.15.50:0/3783566945 shutdown_connections 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.530+0000 7f0058200700 1 -- 172.21.15.50:0/3783566945 wait complete. 2024-05-10T21:13:30.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.531+0000 7f0058200700 1 Processor -- start 2024-05-10T21:13:30.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 -- start start 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0050107df0 0x7f005019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 0x7f00501a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f00501133c0 con 0x7f0050108740 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0050113240 con 0x7f00501034c0 2024-05-10T21:13:30.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.533+0000 7f0058200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0050113540 con 0x7f0050107df0 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:38146/0 (socket says 172.21.15.50:38146) 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 -- 172.21.15.50:0/2649920324 learned_addr learned my addr 172.21.15.50:0/2649920324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 -- 172.21.15.50:0/2649920324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0050107df0 msgr2=0x7f005019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0054e00700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0050107df0 0x7f005019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0050107df0 0x7f005019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0056200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 0x7f00501a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 -- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 msgr2=0x7f00501a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 0x7f00501a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.534+0000 7f0055800700 1 -- 172.21.15.50:0/2649920324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00501a1ef0 con 0x7f00501034c0 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.535+0000 7f0056200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 0x7f00501a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.535+0000 7f0055800700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0044009990 tx=0x7f00440047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.535+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044027070 con 0x7f00501034c0 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.535+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0044004020 con 0x7f00501034c0 2024-05-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.535+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044022db0 con 0x7f00501034c0 2024-05-10T21:13:30.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.537+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00501a2180 con 0x7f00501034c0 2024-05-10T21:13:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.537+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501a2610 con 0x7f00501034c0 2024-05-10T21:13:30.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.538+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00500512c0 con 0x7f00501034c0 2024-05-10T21:13:30.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.541+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0044022470 con 0x7f00501034c0 2024-05-10T21:13:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.543+0000 7f004e000700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 0x7f00380b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:30.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.543+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f00440e3880 con 0x7f00501034c0 2024-05-10T21:13:30.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.544+0000 7f0054e00700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 0x7f00380b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:30.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.545+0000 7f0054e00700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 0x7f00380b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f003c005950 tx=0x7f003c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:30.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.549+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0044030070 con 0x7f00501034c0 2024-05-10T21:13:30.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.997+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00501a9c20 con 0x7f00501034c0 2024-05-10T21:13:30.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:30.998+0000 7f004e000700 1 -- 172.21.15.50:0/2649920324 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11858 (secure 0 0 0) 0x7f0044013070 con 0x7f00501034c0 2024-05-10T21:13:30.996 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:13:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:11:36.879959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:13:30.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 msgr2=0x7f00380b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:30.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 0x7f00380b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f003c005950 tx=0x7f003c0058e0 comp rx=0 tx=0).stop 2024-05-10T21:13:30.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 msgr2=0x7f005019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:30.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0044009990 tx=0x7f00440047b0 comp rx=0 tx=0).stop 2024-05-10T21:13:31.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 shutdown_connections 2024-05-10T21:13:31.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f00380b29d0 0x7f00380b4ee0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:31.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00501034c0 0x7f005019d060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:31.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0050108740 0x7f00501a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:31.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 --2- 172.21.15.50:0/2649920324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0050107df0 0x7f005019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:31.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.002+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 >> 172.21.15.50:0/2649920324 conn(0x7f00500fedc0 msgr2=0x7f00501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:31.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.003+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 shutdown_connections 2024-05-10T21:13:31.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:31.003+0000 7f0058200700 1 -- 172.21.15.50:0/2649920324 wait complete. 2024-05-10T21:13:31.003 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 31 2024-05-10T21:13:31.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:30 smithi092 ceph-mon[34415]: pgmap v600: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 92 KiB/s wr, 12 op/s 2024-05-10T21:13:31.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:30 smithi142 ceph-mon[34846]: pgmap v600: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 92 KiB/s wr, 12 op/s 2024-05-10T21:13:31.381 INFO:tasks.workunit.client.0.smithi050.stdout: 55 85098 64 52115 177 33457 46 2024-05-10T21:13:31.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:30 smithi050 ceph-mon[29546]: pgmap v600: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 92 KiB/s wr, 12 op/s 2024-05-10T21:13:32.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0' 2024-05-10T21:13:32.019 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db / --timeout 60 --expiration 85.15 --await-for 42.0 2024-05-10T21:13:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:31 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2649920324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:32.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:31 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2649920324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:32.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:31 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2649920324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:33.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:32 smithi092 ceph-mon[34415]: pgmap v601: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 99 KiB/s wr, 17 op/s 2024-05-10T21:13:33.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:32 smithi142 ceph-mon[34846]: pgmap v601: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 99 KiB/s wr, 17 op/s 2024-05-10T21:13:33.474 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:32 smithi050 ceph-mon[29546]: pgmap v601: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 99 KiB/s wr, 17 op/s 2024-05-10T21:13:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:34 smithi092 ceph-mon[34415]: pgmap v602: 97 pgs: 97 active+clean; 234 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 314 KiB/s wr, 34 op/s 2024-05-10T21:13:35.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:34 smithi142 ceph-mon[34846]: pgmap v602: 97 pgs: 97 active+clean; 234 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 314 KiB/s wr, 34 op/s 2024-05-10T21:13:35.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:34 smithi050 ceph-mon[29546]: pgmap v602: 97 pgs: 97 active+clean; 234 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 314 KiB/s wr, 34 op/s 2024-05-10T21:13:37.276 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:37.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:36 smithi092 ceph-mon[34415]: pgmap v603: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 797 KiB/s wr, 57 op/s 2024-05-10T21:13:37.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:36 smithi142 ceph-mon[34846]: pgmap v603: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 797 KiB/s wr, 57 op/s 2024-05-10T21:13:37.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:36 smithi050 ceph-mon[29546]: pgmap v603: 97 pgs: 97 active+clean; 239 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 797 KiB/s wr, 57 op/s 2024-05-10T21:13:39.254 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:38 smithi050 ceph-mon[29546]: pgmap v604: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-10T21:13:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:38 smithi092 ceph-mon[34415]: pgmap v604: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-10T21:13:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:38 smithi142 ceph-mon[34846]: pgmap v604: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.489+0000 7f7e07000640 1 -- 172.21.15.50:0/632101872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e001121f0 msgr2=0x7f7e00114610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 --2- 172.21.15.50:0/632101872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e001121f0 0x7f7e00114610 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009930 tx=0x7f7df402a440 comp rx=0 tx=0).stop 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 -- 172.21.15.50:0/632101872 shutdown_connections 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 --2- 172.21.15.50:0/632101872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e001121f0 0x7f7e00114610 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 --2- 172.21.15.50:0/632101872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e00111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 --2- 172.21.15.50:0/632101872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e0010a0f0 0x7f7e0010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.490+0000 7f7e07000640 1 -- 172.21.15.50:0/632101872 >> 172.21.15.50:0/632101872 conn(0x7f7e00070460 msgr2=0x7f7e00072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.492+0000 7f7e07000640 1 -- 172.21.15.50:0/632101872 shutdown_connections 2024-05-10T21:13:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.492+0000 7f7e07000640 1 -- 172.21.15.50:0/632101872 wait complete. 2024-05-10T21:13:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.495+0000 7f7e07000640 1 Processor -- start 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 -- start start 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 0x7f7e0011a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e0011af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e001121f0 0x7f7e00117540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c220 con 0x7f7e0010a0f0 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c0a0 con 0x7f7e0010f890 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.499+0000 7f7e07000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c3a0 con 0x7f7e001121f0 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.502+0000 7f7e06600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e001121f0 0x7f7e00117540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.502+0000 7f7e05200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e0011af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.502+0000 7f7e05200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e0011af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55724/0 (socket says 172.21.15.50:55724) 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.502+0000 7f7e05200640 1 -- 172.21.15.50:0/2451208724 learned_addr learned my addr 172.21.15.50:0/2451208724 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.503+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 0x7f7e0011a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.504+0000 7f7e05c00640 1 -- 172.21.15.50:0/2451208724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e001121f0 msgr2=0x7f7e00117540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.504+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e001121f0 0x7f7e00117540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.504+0000 7f7e05c00640 1 -- 172.21.15.50:0/2451208724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 msgr2=0x7f7e0011af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.504+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e0011af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.504+0000 7f7e05c00640 1 -- 172.21.15.50:0/2451208724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e00117cd0 con 0x7f7e0010a0f0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.505+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 0x7f7e0011a9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005fd0 tx=0x7f7df800c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.505+0000 7f7dfe000640 1 -- 172.21.15.50:0/2451208724 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800ee50 con 0x7f7e0010a0f0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.506+0000 7f7dfe000640 1 -- 172.21.15.50:0/2451208724 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df8033930 con 0x7f7e0010a0f0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.506+0000 7f7dfe000640 1 -- 172.21.15.50:0/2451208724 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df803ca30 con 0x7f7e0010a0f0 2024-05-10T21:13:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.507+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e00117fc0 con 0x7f7e0010a0f0 2024-05-10T21:13:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.508+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e001a49a0 con 0x7f7e0010a0f0 2024-05-10T21:13:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.527+0000 7f7e0f592740 1 Processor -- start 2024-05-10T21:13:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.529+0000 7f7e0f592740 1 -- start start 2024-05-10T21:13:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.529+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.531+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008dd7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.531+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008dde770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.531+0000 7f7e0f592740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560008d3b550 con 0x560008a144d0 2024-05-10T21:13:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.531+0000 7f7e0f592740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560008ce0bc0 con 0x560008a0c2f0 2024-05-10T21:13:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.531+0000 7f7e0f592740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560008d37510 con 0x560008a351b0 2024-05-10T21:13:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47582/0 (socket says 172.21.15.50:47582) 2024-05-10T21:13:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 -- 172.21.15.50:0/3887416534 learned_addr learned my addr 172.21.15.50:0/3887416534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e05200640 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008dde770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 -- 172.21.15.50:0/3887416534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 msgr2=0x560008dd7870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:13:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008dd7870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 -- 172.21.15.50:0/3887416534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 msgr2=0x560008dde770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008dde770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.532+0000 7f7e06600640 1 -- 172.21.15.50:0/3887416534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560008a4d210 con 0x560008a351b0 2024-05-10T21:13:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.533+0000 7f7e05200640 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008dde770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.533+0000 7f7e06600640 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df4004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd9979e597c16248 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.534+0000 7f7dfe000640 1 -- 172.21.15.50:0/2451208724 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7df803cbd0 con 0x7f7e0010a0f0 2024-05-10T21:13:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.536+0000 7f7dfe000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 0x7f7de80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.537+0000 7f7dfe000640 1 -- 172.21.15.50:0/2451208724 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7df81040f0 con 0x7f7e0010a0f0 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.536+0000 7f7df2a00640 1 -- 172.21.15.50:0/3887416534 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4035030 con 0x560008a351b0 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.538+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 0x7f7de80b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.539+0000 7f7df2a00640 1 -- 172.21.15.50:0/3887416534 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df40054f0 con 0x560008a351b0 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.539+0000 7f7df2a00640 1 -- 172.21.15.50:0/3887416534 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4003850 con 0x560008a351b0 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.540+0000 7f7e0f592740 1 -- 172.21.15.50:0/3887416534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 msgr2=0x560008dd72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.540+0000 7f7e0f592740 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df4004310 comp rx=0 tx=0).stop 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e05c00640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 0x7f7de80b4fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7df80cb670 tx=0x7f7df800cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e0f592740 1 -- 172.21.15.50:0/3887416534 shutdown_connections 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e0f592740 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008dde770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e0f592740 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008dd7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e0f592740 1 --2- 172.21.15.50:0/3887416534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008dd72f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.551+0000 7f7e0f592740 1 -- 172.21.15.50:0/3887416534 >> 172.21.15.50:0/3887416534 conn(0x560008a29300 msgr2=0x560008dd2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.559+0000 7f7e0f592740 1 -- 172.21.15.50:0/3887416534 shutdown_connections 2024-05-10T21:13:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.559+0000 7f7e0f592740 1 -- 172.21.15.50:0/3887416534 wait complete. 2024-05-10T21:13:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.560+0000 7f7e0f592740 1 -- start start 2024-05-10T21:13:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.560+0000 7f7e0f592740 1 Processor -- start 2024-05-10T21:13:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.562+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.565+0000 7f7e06600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.565+0000 7f7e06600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48882/0 (socket says 172.21.15.50:48882) 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.565+0000 7f7e06600640 1 -- 172.21.15.50:0/2480695796 learned_addr learned my addr 172.21.15.50:0/2480695796 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.565+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a144d0 0x560008de1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560008ddf160 con 0x560008a0c2f0 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560008cf4660 con 0x560008a144d0 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560008ddf2e0 con 0x560008a351b0 2024-05-10T21:13:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e05200640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a144d0 0x560008de1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.566+0000 7f7e0f592740 10 client.0 _ll_register_callbacks cb 0x560008cbe400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.568+0000 7f7e06600640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.568+0000 7f7e06600640 1 -- 172.21.15.50:0/2480695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 msgr2=0x560008de8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.568+0000 7f7e06600640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.568+0000 7f7e06600640 1 -- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a144d0 msgr2=0x560008de1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.568+0000 7f7e06600640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a144d0 0x560008de1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.569+0000 7f7e06600640 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560008b19620 con 0x560008a0c2f0 2024-05-10T21:13:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.569+0000 7f7e06600640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7df40092a0 tx=0x7f7df402a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b36ddf4f170e563 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.571+0000 7f7de7400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.572+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df404b040 con 0x560008a0c2f0 2024-05-10T21:13:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.572+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7df4005260 con 0x560008a0c2f0 2024-05-10T21:13:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.576+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df40054f0 con 0x560008a0c2f0 2024-05-10T21:13:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.592+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7df4035040 con 0x560008a0c2f0 2024-05-10T21:13:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.593+0000 7f7e0f592740 10 client.16167 fetch_fsmap learned FSMap version 31 2024-05-10T21:13:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.593+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560008deb100 con 0x560008a0c2f0 2024-05-10T21:13:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.594+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df40054f0 con 0x560008a0c2f0 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.594+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7df4031860 con 0x560008a0c2f0 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 10 client.16167 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 20 client.16167 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089f3d80 0x560008dee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 4 client.16167 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560008d49160 con 0x5600089f3d80 2024-05-10T21:13:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.596+0000 7f7e05200640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089f3d80 0x560008dee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.603+0000 7f7e05200640 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089f3d80 0x560008dee460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532c37a5ab432240 server_cookie=66af5b476a67aabe in_seq=0 out_seq=0 2024-05-10T21:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.607+0000 7f7de7400640 10 client.16167 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.607+0000 7f7de7400640 1 -- 172.21.15.50:0/2480695796 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560008d49160 con 0x5600089f3d80 2024-05-10T21:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.607+0000 7f7de7400640 10 client.16167 handle_command_reply: tid=0 2024-05-10T21:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.607+0000 7f7e0f592740 1 client.16167 shutdown 2024-05-10T21:13:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.612+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 msgr2=0x560008de0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.612+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7df40092a0 tx=0x7f7df402a940 comp rx=0 tx=0).stop 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.612+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 shutdown_connections 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.612+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089f3d80 0x560008dee460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a144d0 0x560008de1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2480695796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de0de0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 >> 172.21.15.50:0/2480695796 conn(0x560008a29300 msgr2=0x560008dd1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7de7400640 0 client.16167 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 shutdown_connections 2024-05-10T21:13:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.613+0000 7f7e0f592740 1 -- 172.21.15.50:0/2480695796 wait complete. 2024-05-10T21:13:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.614+0000 7f7e0f592740 20 client.16167 trim_cache size 0 max 16384 2024-05-10T21:13:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.965+0000 7f7e0f592740 1 Processor -- start 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.989+0000 7f7e0f592740 1 -- start start 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008de03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008de0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001b3f10 con 0x560008a144d0 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001b1720 con 0x560008a0c2f0 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e0f592740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001ab680 con 0x560008a351b0 2024-05-10T21:13:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e05200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008de0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.990+0000 7f7e05200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008de0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48898/0 (socket says 172.21.15.50:48898) 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.991+0000 7f7e05200640 1 -- 172.21.15.50:0/2013541783 learned_addr learned my addr 172.21.15.50:0/2013541783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.991+0000 7f7e06600640 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008de03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.992+0000 7f7e06600640 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.993+0000 7f7e06600640 1 -- 172.21.15.50:0/2013541783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 msgr2=0x560008de0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.993+0000 7f7e06600640 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.993+0000 7f7e06600640 1 -- 172.21.15.50:0/2013541783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 msgr2=0x560008de0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.993+0000 7f7e06600640 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008de0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:39.994+0000 7f7e06600640 1 -- 172.21.15.50:0/2013541783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600089f3fb0 con 0x560008a0c2f0 2024-05-10T21:13:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.011+0000 7f7e06600640 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008de03c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7df40041a0 tx=0x7f7df4004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d695edf6c022aed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.015+0000 7f7df2000640 1 -- 172.21.15.50:0/2013541783 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df405f040 con 0x560008a0c2f0 2024-05-10T21:13:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.015+0000 7f7df2000640 1 -- 172.21.15.50:0/2013541783 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df4060040 con 0x560008a0c2f0 2024-05-10T21:13:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.015+0000 7f7df2000640 1 -- 172.21.15.50:0/2013541783 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df40054c0 con 0x560008a0c2f0 2024-05-10T21:13:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.038+0000 7f7e0f592740 1 -- 172.21.15.50:0/2013541783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 msgr2=0x560008de03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.038+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008de03c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7df40041a0 tx=0x7f7df4004310 comp rx=0 tx=0).stop 2024-05-10T21:13:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.080+0000 7f7e0f592740 1 -- 172.21.15.50:0/2013541783 shutdown_connections 2024-05-10T21:13:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.081+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a351b0 0x560008de0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.081+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a144d0 0x560008de0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.081+0000 7f7e0f592740 1 --2- 172.21.15.50:0/2013541783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a0c2f0 0x560008de03c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.081+0000 7f7e0f592740 1 -- 172.21.15.50:0/2013541783 >> 172.21.15.50:0/2013541783 conn(0x560008a29300 msgr2=0x560008ddf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.101+0000 7f7e0f592740 1 -- 172.21.15.50:0/2013541783 shutdown_connections 2024-05-10T21:13:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.101+0000 7f7e0f592740 1 -- 172.21.15.50:0/2013541783 wait complete. 2024-05-10T21:13:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.103+0000 7f7e0f592740 1 -- start start 2024-05-10T21:13:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.104+0000 7f7e0f592740 1 Processor -- start 2024-05-10T21:13:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.106+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.108+0000 7f7e06600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.108+0000 7f7e06600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48902/0 (socket says 172.21.15.50:48902) 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 0x560008de6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e06600640 1 -- 172.21.15.50:0/779100297 learned_addr learned my addr 172.21.15.50:0/779100297 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e05200640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 0x560008de6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a351b0 0x560008de6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001abcc0 con 0x560008a0c2f0 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001b1070 con 0x560008a351b0 2024-05-10T21:13:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001ac390 con 0x560008a144d0 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.109+0000 7f7e0f592740 10 client.0 _ll_register_callbacks cb 0x7f7df4004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.110+0000 7f7e06600640 1 -- 172.21.15.50:0/779100297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 msgr2=0x560008de6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.110+0000 7f7e06600640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 0x560008de6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.110+0000 7f7e06600640 1 -- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a351b0 msgr2=0x560008de6c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.110+0000 7f7e06600640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a351b0 0x560008de6c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.110+0000 7f7e06600640 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560008e54790 con 0x560008a0c2f0 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.111+0000 7f7e06600640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7df4052040 tx=0x7f7df4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95396ec9864d875b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.112+0000 7f7e05200640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 0x560008de6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.133+0000 7f7de7400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.133+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df405f070 con 0x560008a0c2f0 2024-05-10T21:13:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.133+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7df4005470 con 0x560008a0c2f0 2024-05-10T21:13:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.163+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df4060040 con 0x560008a0c2f0 2024-05-10T21:13:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.172+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7df4009780 con 0x560008a0c2f0 2024-05-10T21:13:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.179+0000 7f7e0f592740 10 client.16173 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.179+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560008e55a20 con 0x560008a0c2f0 2024-05-10T21:13:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.180+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df4060040 con 0x560008a0c2f0 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.180+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f7df4085070 con 0x560008a0c2f0 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 10 client.16173 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 20 client.16173 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089c91c0 0x560008e583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 4 client.16173 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:13:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.181+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560008d42fd0 con 0x5600089c91c0 2024-05-10T21:13:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.186+0000 7f7e05200640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089c91c0 0x560008e583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.190+0000 7f7e05200640 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089c91c0 0x560008e583d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4133e3e266d21f2 server_cookie=ab76f78af1126c40 in_seq=0 out_seq=0 2024-05-10T21:13:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.190+0000 7f7de7400640 10 client.16173 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.604+0000 7f7de7400640 1 -- 172.21.15.50:0/779100297 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560008d42fd0 con 0x5600089c91c0 2024-05-10T21:13:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.604+0000 7f7de7400640 10 client.16173 handle_command_reply: tid=0 2024-05-10T21:13:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.606+0000 7f7e0f592740 1 client.16173 shutdown 2024-05-10T21:13:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.610+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 msgr2=0x560008de6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.610+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7df4052040 tx=0x7f7df4000bc0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 shutdown_connections 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600089c91c0 0x560008e583d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560008a144d0 0x560008de6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560008a351b0 0x560008de6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 --2- 172.21.15.50:0/779100297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560008a0c2f0 0x560008de6210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 >> 172.21.15.50:0/779100297 conn(0x560008a29300 msgr2=0x560008dd5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.619+0000 7f7de7400640 0 client.16173 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.620+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 shutdown_connections 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.621+0000 7f7e0f592740 1 -- 172.21.15.50:0/779100297 wait complete. 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.621+0000 7f7e0f592740 20 client.16173 trim_cache size 0 max 16384 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:13:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.660+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 msgr2=0x7f7de80b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.660+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 0x7f7de80b4fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7df80cb670 tx=0x7f7df800cf50 comp rx=0 tx=0).stop 2024-05-10T21:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.660+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 msgr2=0x7f7e0011a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.660+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 0x7f7e0011a9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005fd0 tx=0x7f7df800c640 comp rx=0 tx=0).stop 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 shutdown_connections 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7de80b2ad0 0x7f7de80b4fe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e001121f0 0x7f7e00117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e0010f890 0x7f7e0011af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 --2- 172.21.15.50:0/2451208724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e0010a0f0 0x7f7e0011a9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.661+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 >> 172.21.15.50:0/2451208724 conn(0x7f7e00070460 msgr2=0x7f7e00072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.668+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 shutdown_connections 2024-05-10T21:13:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:40.670+0000 7f7e07000640 1 -- 172.21.15.50:0/2451208724 wait complete. 2024-05-10T21:13:41.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:41 smithi050 ceph-mon[29546]: pgmap v605: 97 pgs: 97 active+clean; 243 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-10T21:13:41.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:41 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:13:41.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:41 smithi092 ceph-mon[34415]: pgmap v605: 97 pgs: 97 active+clean; 243 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-10T21:13:41.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:41 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:13:41.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:41 smithi142 ceph-mon[34846]: pgmap v605: 97 pgs: 97 active+clean; 243 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-10T21:13:41.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:41 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:13:41.403 INFO:tasks.workunit.client.0.smithi050.stdout: 67 544522 683 344124 475 179348 91 2024-05-10T21:13:42.212 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:13:43.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:43 smithi050 ceph-mon[29546]: pgmap v606: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T21:13:43.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:43 smithi092 ceph-mon[34415]: pgmap v606: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T21:13:43.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:43 smithi142 ceph-mon[34846]: pgmap v606: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T21:13:43.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.921+0000 7fb648800700 1 -- 172.21.15.50:0/1727334393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb640116bf0 msgr2=0x7fb640119010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.922+0000 7fb648800700 1 --2- 172.21.15.50:0/1727334393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb640116bf0 0x7fb640119010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb630013ef0 tx=0x7fb6300097f0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.922+0000 7fb648800700 1 -- 172.21.15.50:0/1727334393 shutdown_connections 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.922+0000 7fb648800700 1 --2- 172.21.15.50:0/1727334393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640119550 0x7fb64011b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.923+0000 7fb648800700 1 --2- 172.21.15.50:0/1727334393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb64010ae90 0x7fb64010b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.924+0000 7fb648800700 1 --2- 172.21.15.50:0/1727334393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb640116bf0 0x7fb640119010 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.924+0000 7fb648800700 1 -- 172.21.15.50:0/1727334393 >> 172.21.15.50:0/1727334393 conn(0x7fb64006f990 msgr2=0x7fb64006fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:43.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.925+0000 7fb648800700 1 -- 172.21.15.50:0/1727334393 shutdown_connections 2024-05-10T21:13:43.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.925+0000 7fb648800700 1 -- 172.21.15.50:0/1727334393 wait complete. 2024-05-10T21:13:43.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.926+0000 7fb648800700 1 Processor -- start 2024-05-10T21:13:43.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.929+0000 7fb648800700 1 -- start start 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.934+0000 7fb648800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.934+0000 7fb648800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 0x7fb6401a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.934+0000 7fb648800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 0x7fb6401a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb648800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb640111060 con 0x7fb640116bf0 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb648800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb640110ee0 con 0x7fb64010ae90 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb648800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401111e0 con 0x7fb640119550 2024-05-10T21:13:43.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb645e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb645400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 0x7fb6401a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb645e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56078/0 (socket says 172.21.15.50:56078) 2024-05-10T21:13:43.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.935+0000 7fb645e00700 1 -- 172.21.15.50:0/488009898 learned_addr learned my addr 172.21.15.50:0/488009898 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.936+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 0x7fb6401a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.937+0000 7fb646800700 1 -- 172.21.15.50:0/488009898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 msgr2=0x7fb6401a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.937+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.937+0000 7fb646800700 1 -- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 msgr2=0x7fb6401a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.938+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 0x7fb6401a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.938+0000 7fb646800700 1 -- 172.21.15.50:0/488009898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6401a1b90 con 0x7fb640119550 2024-05-10T21:13:43.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.938+0000 7fb645400700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 0x7fb6401a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:43.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.938+0000 7fb645e00700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:43.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.938+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 0x7fb6401a1460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00be00 tx=0x7fb63c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:43.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.939+0000 7fb63aa00700 1 -- 172.21.15.50:0/488009898 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c019070 con 0x7fb640119550 2024-05-10T21:13:43.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.939+0000 7fb63aa00700 1 -- 172.21.15.50:0/488009898 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c00cc10 con 0x7fb640119550 2024-05-10T21:13:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.939+0000 7fb63aa00700 1 -- 172.21.15.50:0/488009898 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c012580 con 0x7fb640119550 2024-05-10T21:13:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.939+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6401a1e80 con 0x7fb640119550 2024-05-10T21:13:43.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.939+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6401ae050 con 0x7fb640119550 2024-05-10T21:13:43.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.959+0000 7fb65742fb80 1 Processor -- start 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.959+0000 7fb65742fb80 1 -- start start 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c421fd0 0x55ef0c59fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c4c1bb0 con 0x55ef0c421fd0 2024-05-10T21:13:43.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c45f8f0 con 0x55ef0c5a49b0 2024-05-10T21:13:43.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb65742fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c4bdb70 con 0x55ef0c5a01d0 2024-05-10T21:13:43.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.960+0000 7fb63aa00700 1 -- 172.21.15.50:0/488009898 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb63c01c070 con 0x7fb640119550 2024-05-10T21:13:43.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.963+0000 7fb645e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.964+0000 7fb645400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c421fd0 0x55ef0c59fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.964+0000 7fb645e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53072/0 (socket says 172.21.15.50:53072) 2024-05-10T21:13:43.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.964+0000 7fb645e00700 1 -- 172.21.15.50:0/658997485 learned_addr learned my addr 172.21.15.50:0/658997485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.964+0000 7fb63aa00700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 0x7fb6280b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.965+0000 7fb63aa00700 1 -- 172.21.15.50:0/488009898 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb63c0e16f0 con 0x7fb640119550 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.966+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 0x7fb6280b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.966+0000 7fb645400700 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645e00700 1 -- 172.21.15.50:0/658997485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 msgr2=0x55ef0c5a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645e00700 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645e00700 1 -- 172.21.15.50:0/658997485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c421fd0 msgr2=0x55ef0c59fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645e00700 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c421fd0 0x55ef0c59fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645e00700 1 -- 172.21.15.50:0/658997485 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0c40ea60 con 0x55ef0c5a01d0 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.967+0000 7fb645400700 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.968+0000 7fb646800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 0x7fb6280b4f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb63c014c70 tx=0x7fb63c014bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:43.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.969+0000 7fb645e00700 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb63400aa80 tx=0x7fb63400ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e65eb903901203f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:43.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.970+0000 7fb62f400700 1 -- 172.21.15.50:0/658997485 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63400ca90 con 0x55ef0c5a01d0 2024-05-10T21:13:43.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.970+0000 7fb62f400700 1 -- 172.21.15.50:0/658997485 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63400cc30 con 0x55ef0c5a01d0 2024-05-10T21:13:43.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.974+0000 7fb65742fb80 1 -- 172.21.15.50:0/658997485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 msgr2=0x55ef0c5a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:43.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.974+0000 7fb65742fb80 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb63400aa80 tx=0x7fb63400ad90 comp rx=0 tx=0).stop 2024-05-10T21:13:43.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 -- 172.21.15.50:0/658997485 shutdown_connections 2024-05-10T21:13:43.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c421fd0 0x55ef0c59fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 --2- 172.21.15.50:0/658997485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a4470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 -- 172.21.15.50:0/658997485 >> 172.21.15.50:0/658997485 conn(0x55ef0c467100 msgr2=0x55ef0c59aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 -- 172.21.15.50:0/658997485 shutdown_connections 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.979+0000 7fb65742fb80 1 -- 172.21.15.50:0/658997485 wait complete. 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.980+0000 7fb65742fb80 1 -- start start 2024-05-10T21:13:43.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.980+0000 7fb65742fb80 1 Processor -- start 2024-05-10T21:13:43.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c5a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5aa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:43.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c5bd9c0 con 0x55ef0c5a01d0 2024-05-10T21:13:43.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c427cb0 con 0x55ef0c421fd0 2024-05-10T21:13:43.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ef0c5bdb40 con 0x55ef0c5a49b0 2024-05-10T21:13:43.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.983+0000 7fb65742fb80 10 client.0 _ll_register_callbacks cb 0x55ef0c468e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:43.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.988+0000 7fb645400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c5a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.988+0000 7fb645400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c5a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56090/0 (socket says 172.21.15.50:56090) 2024-05-10T21:13:43.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.988+0000 7fb645400700 1 -- 172.21.15.50:0/1385259548 learned_addr learned my addr 172.21.15.50:0/1385259548 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:43.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.988+0000 7fb645400700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.991+0000 7fb645e00700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.994+0000 7fb645e00700 1 -- 172.21.15.50:0/1385259548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 msgr2=0x55ef0c5aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.994+0000 7fb645e00700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5aa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.994+0000 7fb645e00700 1 -- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 msgr2=0x55ef0c5a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.994+0000 7fb645e00700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c5a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.994+0000 7fb645e00700 1 -- 172.21.15.50:0/1385259548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0c4875b0 con 0x55ef0c5a01d0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.995+0000 7fb645e00700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a9b00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb634011cc0 tx=0x7fb634011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedc3b3b1dfbc53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.995+0000 7fb623e00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.995+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634029060 con 0x55ef0c5a01d0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.996+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb634011600 con 0x55ef0c5a01d0 2024-05-10T21:13:43.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.996+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63400cc30 con 0x55ef0c5a01d0 2024-05-10T21:13:43.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.997+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb634011820 con 0x55ef0c5a01d0 2024-05-10T21:13:43.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.997+0000 7fb65742fb80 10 client.16182 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:43.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.997+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef0c5c8490 con 0x55ef0c5a01d0 2024-05-10T21:13:44.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.997+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63400cc30 con 0x55ef0c5a01d0 2024-05-10T21:13:44.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fb634010b80 con 0x55ef0c5a01d0 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 10 client.16182 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 20 client.16182 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c5ca960 0x55ef0c5ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 4 client.16182 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef0c56c350 con 0x55ef0c5ca960 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.998+0000 7fb645400700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c5ca960 0x55ef0c5ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.999+0000 7fb645400700 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c5ca960 0x55ef0c5ccd80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ecfcd7c5e39d566 server_cookie=5ef43180e7ecba49 in_seq=0 out_seq=0 2024-05-10T21:13:44.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:43.999+0000 7fb623e00700 10 client.16182 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb623e00700 1 -- 172.21.15.50:0/1385259548 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef0c56c350 con 0x55ef0c5ca960 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb623e00700 10 client.16182 handle_command_reply: tid=0 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 client.16182 shutdown 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 msgr2=0x55ef0c5a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a9b00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb634011cc0 tx=0x7fb634011b70 comp rx=0 tx=0).stop 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 shutdown_connections 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c5ca960 0x55ef0c5ccd80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c5a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb623e00700 0 client.16182 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1385259548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5a01d0 0x55ef0c5a9b00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.003+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 >> 172.21.15.50:0/1385259548 conn(0x55ef0c467100 msgr2=0x55ef0c59e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.007+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 shutdown_connections 2024-05-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.007+0000 7fb65742fb80 1 -- 172.21.15.50:0/1385259548 wait complete. 2024-05-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.007+0000 7fb65742fb80 20 client.16182 trim_cache size 0 max 16384 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.070+0000 7fb65742fb80 1 Processor -- start 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.072+0000 7fb65742fb80 1 -- start start 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.072+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c59f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.072+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.073+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c5d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.073+0000 7fb65742fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b7a30 con 0x55ef0c5cfdb0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.073+0000 7fb65742fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b5230 con 0x55ef0c421fd0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.073+0000 7fb65742fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b3100 con 0x55ef0c5a49b0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53082/0 (socket says 172.21.15.50:53082) 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 -- 172.21.15.50:0/1908014557 learned_addr learned my addr 172.21.15.50:0/1908014557 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 -- 172.21.15.50:0/1908014557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 msgr2=0x55ef0c59f410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c59f410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645e00700 1 -- 172.21.15.50:0/1908014557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 msgr2=0x55ef0c5d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.074+0000 7fb645400700 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c5d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.075+0000 7fb645e00700 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c5d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.075+0000 7fb645e00700 1 -- 172.21.15.50:0/1908014557 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0c5aa3c0 con 0x55ef0c5a49b0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.075+0000 7fb645e00700 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb63400a9d0 tx=0x7fb634011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63acee604df9b63c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.077+0000 7fb62ea00700 1 -- 172.21.15.50:0/1908014557 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634029040 con 0x55ef0c5a49b0 2024-05-10T21:13:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.077+0000 7fb62ea00700 1 -- 172.21.15.50:0/1908014557 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb634012e80 con 0x55ef0c5a49b0 2024-05-10T21:13:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.077+0000 7fb62ea00700 1 -- 172.21.15.50:0/1908014557 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634033080 con 0x55ef0c5a49b0 2024-05-10T21:13:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.078+0000 7fb65742fb80 1 -- 172.21.15.50:0/1908014557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 msgr2=0x55ef0c5a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.078+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb63400a9d0 tx=0x7fb634011be0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 -- 172.21.15.50:0/1908014557 shutdown_connections 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c421fd0 0x55ef0c59f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c5d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 --2- 172.21.15.50:0/1908014557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c5a49b0 0x55ef0c5a9e80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 -- 172.21.15.50:0/1908014557 >> 172.21.15.50:0/1908014557 conn(0x55ef0c467100 msgr2=0x55ef0c594d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:44.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 -- 172.21.15.50:0/1908014557 shutdown_connections 2024-05-10T21:13:44.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.081+0000 7fb65742fb80 1 -- 172.21.15.50:0/1908014557 wait complete. 2024-05-10T21:13:44.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.082+0000 7fb65742fb80 1 -- start start 2024-05-10T21:13:44.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.082+0000 7fb65742fb80 1 Processor -- start 2024-05-10T21:13:44.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.088+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 0x55ef0c62ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.088+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c62f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb645400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 0x55ef0c62ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb65742fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c629100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb645400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 0x55ef0c62ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53098/0 (socket says 172.21.15.50:53098) 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb65742fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b3750 con 0x55ef0c5cfdb0 2024-05-10T21:13:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb645400700 1 -- 172.21.15.50:0/3877413457 learned_addr learned my addr 172.21.15.50:0/3877413457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b4b60 con 0x55ef0c5a49b0 2024-05-10T21:13:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401b3dc0 con 0x55ef0c421fd0 2024-05-10T21:13:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.089+0000 7fb65742fb80 10 client.0 _ll_register_callbacks cb 0x55ef0c468e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.090+0000 7fb645e00700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c62f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.091+0000 7fb645400700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c629100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.092+0000 7fb645e00700 1 -- 172.21.15.50:0/3877413457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 msgr2=0x55ef0c62ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:44.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.092+0000 7fb645e00700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 0x55ef0c62ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.092+0000 7fb645e00700 1 -- 172.21.15.50:0/3877413457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 msgr2=0x55ef0c629100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:44.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.093+0000 7fb645e00700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c629100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:44.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.093+0000 7fb645e00700 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0c629770 con 0x55ef0c5a49b0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.093+0000 7fb645400700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c629100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.096+0000 7fb645e00700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c62f240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb63400a200 tx=0x7fb63400cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c6261e7eca5c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.097+0000 7fb623e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.097+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634033070 con 0x55ef0c5a49b0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.097+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb634003890 con 0x55ef0c5a49b0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.098+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb634012e80 con 0x55ef0c5a49b0 2024-05-10T21:13:44.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.098+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb634029040 con 0x55ef0c5a49b0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.098+0000 7fb65742fb80 10 client.26324 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.098+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef0c6363b0 con 0x55ef0c5a49b0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.099+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb634012e80 con 0x55ef0c5a49b0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.099+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fb63402d750 con 0x55ef0c5a49b0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.099+0000 7fb65742fb80 10 client.26324 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.099+0000 7fb65742fb80 10 client.26324 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.099+0000 7fb65742fb80 20 client.26324 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.100+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c6369e0 0x55ef0c638e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.100+0000 7fb65742fb80 4 client.26324 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}] 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.100+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.15, "await_for": 42.0}) -- 0x55ef0c4cf590 con 0x55ef0c6369e0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.100+0000 7fb645400700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c6369e0 0x55ef0c638e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:44.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.101+0000 7fb645400700 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c6369e0 0x55ef0c638e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ba617ac7c59ab0 server_cookie=5aa66282d882e673 in_seq=0 out_seq=0 2024-05-10T21:13:44.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:44.103+0000 7fb623e00700 10 client.26324 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:44.225 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:45.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:45 smithi092 ceph-mon[34415]: pgmap v607: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-05-10T21:13:45.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:45 smithi142 ceph-mon[34846]: pgmap v607: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-05-10T21:13:45.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:45 smithi050 ceph-mon[29546]: pgmap v607: 97 pgs: 97 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-05-10T21:13:47.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:47 smithi092 ceph-mon[34415]: pgmap v608: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-05-10T21:13:47.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:47 smithi142 ceph-mon[34846]: pgmap v608: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-05-10T21:13:47.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:47 smithi050 ceph-mon[29546]: pgmap v608: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-05-10T21:13:48.069 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:48 smithi050 ceph-mon[29546]: pgmap v609: 97 pgs: 97 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-10T21:13:48.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 -- 172.21.15.50:0/2518323370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c110c00 msgr2=0x7f840c113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 --2- 172.21.15.50:0/2518323370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c110c00 0x7f840c113020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8404009f90 tx=0x7f840400b330 comp rx=0 tx=0).stop 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 -- 172.21.15.50:0/2518323370 shutdown_connections 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 --2- 172.21.15.50:0/2518323370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c075b00 0x7f840c075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 --2- 172.21.15.50:0/2518323370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c110c00 0x7f840c113020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 --2- 172.21.15.50:0/2518323370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 -- 172.21.15.50:0/2518323370 >> 172.21.15.50:0/2518323370 conn(0x7f840c100040 msgr2=0x7f840c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 -- 172.21.15.50:0/2518323370 shutdown_connections 2024-05-10T21:13:48.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.092+0000 7f8411200700 1 -- 172.21.15.50:0/2518323370 wait complete. 2024-05-10T21:13:48.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.094+0000 7f8411200700 1 Processor -- start 2024-05-10T21:13:48.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.094+0000 7f8411200700 1 -- start start 2024-05-10T21:13:48.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 0x7f840c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f840c116710 con 0x7f840c076450 2024-05-10T21:13:48.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f840c116590 con 0x7f840c110c00 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8411200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f840c116890 con 0x7f840c075b00 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8409e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8409e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53118/0 (socket says 172.21.15.50:53118) 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8409e00700 1 -- 172.21.15.50:0/147045370 learned_addr learned my addr 172.21.15.50:0/147045370 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f8409400700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.095+0000 7f840a800700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 0x7f840c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 -- 172.21.15.50:0/147045370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 msgr2=0x7f840c19d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 -- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 msgr2=0x7f840c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 -- 172.21.15.50:0/147045370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c1a1f50 con 0x7f840c110c00 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8409400700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8409e00700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f840a800700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 0x7f840c1a18b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f840400b6c0 tx=0x7f840400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8402a00700 1 -- 172.21.15.50:0/147045370 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404021070 con 0x7f840c110c00 2024-05-10T21:13:48.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8402a00700 1 -- 172.21.15.50:0/147045370 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8404007d60 con 0x7f840c110c00 2024-05-10T21:13:48.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8402a00700 1 -- 172.21.15.50:0/147045370 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404005040 con 0x7f840c110c00 2024-05-10T21:13:48.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.096+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c1a21e0 con 0x7f840c110c00 2024-05-10T21:13:48.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.097+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1a26d0 con 0x7f840c110c00 2024-05-10T21:13:48.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.101+0000 7f8402a00700 1 -- 172.21.15.50:0/147045370 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8404024070 con 0x7f840c110c00 2024-05-10T21:13:48.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.104+0000 7f8402a00700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 0x7f83f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.104+0000 7f8402a00700 1 -- 172.21.15.50:0/147045370 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f840400f020 con 0x7f840c110c00 2024-05-10T21:13:48.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.104+0000 7f841fcd9b80 1 Processor -- start 2024-05-10T21:13:48.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.104+0000 7f8409e00700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 0x7f83f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.104+0000 7f841fcd9b80 1 -- start start 2024-05-10T21:13:48.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.105+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.105+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 0x557d55e6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.105+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.105+0000 7f8409e00700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 0x7f83f00b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011550 tx=0x7f83fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53122/0 (socket says 172.21.15.50:53122) 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 0x557d55e6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 0x557d55e6acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56140/0 (socket says 172.21.15.50:56140) 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 -- 172.21.15.50:0/1316467819 learned_addr learned my addr 172.21.15.50:0/1316467819 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f840a800700 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557d55d84a20 con 0x557d55e6b200 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d55d22760 con 0x557d55e71040 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557d55d809e0 con 0x557d55ce4e40 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 -- 172.21.15.50:0/1316467819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 msgr2=0x557d55e6acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 0x557d55e6acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.106+0000 7f8409e00700 1 -- 172.21.15.50:0/1316467819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 msgr2=0x557d55e6b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f8409e00700 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f8409e00700 1 -- 172.21.15.50:0/1316467819 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d55cd18d0 con 0x557d55ce4e40 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f8409e00700 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e6b0 tx=0x7f83fc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df859ced7bf05aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f840a800700 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f83f7400700 1 -- 172.21.15.50:0/1316467819 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc026630 con 0x557d55ce4e40 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f83f7400700 1 -- 172.21.15.50:0/1316467819 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc00ee50 con 0x557d55ce4e40 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.109+0000 7f83f7400700 1 -- 172.21.15.50:0/1316467819 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc025870 con 0x557d55ce4e40 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 msgr2=0x557d55e70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e6b0 tx=0x7f83fc014000 comp rx=0 tx=0).stop 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 shutdown_connections 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55ce4e40 0x557d55e70b00 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e6b0 tx=0x7f83fc014000 comp rx=0 tx=0).stop 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/1316467819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e71040 0x557d55e6acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 >> 172.21.15.50:0/1316467819 conn(0x557d55d29f70 msgr2=0x557d55e5d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 shutdown_connections 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.111+0000 7f841fcd9b80 1 -- 172.21.15.50:0/1316467819 wait complete. 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.112+0000 7f841fcd9b80 1 -- start start 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.112+0000 7f841fcd9b80 1 Processor -- start 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e74f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55e80fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557d55e77b20 con 0x557d55e71040 2024-05-10T21:13:48.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d55ceab20 con 0x557d55e6b200 2024-05-10T21:13:48.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557d55e77ca0 con 0x557d55e75450 2024-05-10T21:13:48.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.113+0000 7f841fcd9b80 10 client.0 _ll_register_callbacks cb 0x557d55d2bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:48.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.114+0000 7f8409e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.114+0000 7f8409e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56142/0 (socket says 172.21.15.50:56142) 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.114+0000 7f8409e00700 1 -- 172.21.15.50:0/2150180357 learned_addr learned my addr 172.21.15.50:0/2150180357 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.114+0000 7f8409400700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e74f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f840a800700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55e80fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f8409e00700 1 -- 172.21.15.50:0/2150180357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 msgr2=0x557d55e80fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f8409e00700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55e80fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f8409e00700 1 -- 172.21.15.50:0/2150180357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 msgr2=0x557d55e74f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f8409e00700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e74f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f8409e00700 1 -- 172.21.15.50:0/2150180357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d55d4a420 con 0x557d55e6b200 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.115+0000 7f840a800700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55e80fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.116+0000 7f8409e00700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f83fc026000 tx=0x7f83fc005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca95112cd718aea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.116+0000 7f83ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.117+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc005560 con 0x557d55e6b200 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.117+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f83fc004710 con 0x557d55e6b200 2024-05-10T21:13:48.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.117+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc00ee50 con 0x557d55e6b200 2024-05-10T21:13:48.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.117+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83fc004910 con 0x557d55e6b200 2024-05-10T21:13:48.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.118+0000 7f841fcd9b80 10 client.26336 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:48.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.118+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d55e82c70 con 0x557d55e6b200 2024-05-10T21:13:48.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.118+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc00ee50 con 0x557d55e6b200 2024-05-10T21:13:48.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f83fc025470 con 0x557d55e6b200 2024-05-10T21:13:48.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:48.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 10 client.26336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:48.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 20 client.26336 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:48.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55e85140 0x557d55e87560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 4 client.26336 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:48.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.119+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d55e2f1c0 con 0x557d55e85140 2024-05-10T21:13:48.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.120+0000 7f8409400700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55e85140 0x557d55e87560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.120+0000 7f8409400700 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55e85140 0x557d55e87560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff8d9e0556ea7e8 server_cookie=30fdd0b68b2c6dce in_seq=0 out_seq=0 2024-05-10T21:13:48.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.120+0000 7f83ebe00700 10 client.26336 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:48.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.121+0000 7f83ebe00700 1 -- 172.21.15.50:0/2150180357 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d55e2f1c0 con 0x557d55e85140 2024-05-10T21:13:48.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.121+0000 7f83ebe00700 10 client.26336 handle_command_reply: tid=0 2024-05-10T21:13:48.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.121+0000 7f841fcd9b80 1 client.26336 shutdown 2024-05-10T21:13:48.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.121+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 msgr2=0x557d55e749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.121+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f83fc026000 tx=0x7f83fc005040 comp rx=0 tx=0).stop 2024-05-10T21:13:48.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 shutdown_connections 2024-05-10T21:13:48.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55e85140 0x557d55e87560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e749d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55e80fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/2150180357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e74f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 >> 172.21.15.50:0/2150180357 conn(0x557d55d29f70 msgr2=0x557d55e69cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f83ebe00700 0 client.26336 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 shutdown_connections 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 1 -- 172.21.15.50:0/2150180357 wait complete. 2024-05-10T21:13:48.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.122+0000 7f841fcd9b80 20 client.26336 trim_cache size 0 max 16384 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.150+0000 7f841fcd9b80 1 Processor -- start 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.150+0000 7f841fcd9b80 1 -- start start 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 0x557d55e74ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1af2f0 con 0x557d55e6b200 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1acaf0 con 0x557d55e75450 2024-05-10T21:13:48.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f840c108240 con 0x557d55e71040 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f8409400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f8409400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53130/0 (socket says 172.21.15.50:53130) 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f8409400700 1 -- 172.21.15.50:0/347034655 learned_addr learned my addr 172.21.15.50:0/347034655 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.151+0000 7f840a800700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 0x557d55e74ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f8409e00700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f840a800700 1 -- 172.21.15.50:0/347034655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 msgr2=0x557d55e74990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f840a800700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f840a800700 1 -- 172.21.15.50:0/347034655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 msgr2=0x557d55e6acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f840a800700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f840a800700 1 -- 172.21.15.50:0/347034655 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d55e6f4d0 con 0x557d55e75450 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.152+0000 7f8409400700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.153+0000 7f840a800700 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 0x557d55e74ed0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f84040aaac0 tx=0x7f8404009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73abb308cc411a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.153+0000 7f83f6a00700 1 -- 172.21.15.50:0/347034655 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040af170 con 0x557d55e75450 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.153+0000 7f83f6a00700 1 -- 172.21.15.50:0/347034655 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84040c1030 con 0x557d55e75450 2024-05-10T21:13:48.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.153+0000 7f83f6a00700 1 -- 172.21.15.50:0/347034655 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840401d890 con 0x557d55e75450 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 -- 172.21.15.50:0/347034655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 msgr2=0x557d55e74ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 0x557d55e74ed0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f84040aaac0 tx=0x7f8404009040 comp rx=0 tx=0).stop 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 -- 172.21.15.50:0/347034655 shutdown_connections 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e75450 0x557d55e74ed0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e6b200 0x557d55e6acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/347034655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e71040 0x557d55e74990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.154+0000 7f841fcd9b80 1 -- 172.21.15.50:0/347034655 >> 172.21.15.50:0/347034655 conn(0x557d55d29f70 msgr2=0x557d55e74280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.155+0000 7f841fcd9b80 1 -- 172.21.15.50:0/347034655 shutdown_connections 2024-05-10T21:13:48.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.155+0000 7f841fcd9b80 1 -- 172.21.15.50:0/347034655 wait complete. 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.155+0000 7f841fcd9b80 1 -- start start 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.155+0000 7f841fcd9b80 1 Processor -- start 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e88b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55ee86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1aafb0 con 0x557d55e71040 2024-05-10T21:13:48.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1ac420 con 0x557d55e6b200 2024-05-10T21:13:48.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1ab680 con 0x557d55e75450 2024-05-10T21:13:48.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f841fcd9b80 10 client.0 _ll_register_callbacks cb 0x557d55d2bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:48.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.156+0000 7f8409e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e88b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e885e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56168/0 (socket says 172.21.15.50:56168) 2024-05-10T21:13:48.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409e00700 1 -- 172.21.15.50:0/3802088158 learned_addr learned my addr 172.21.15.50:0/3802088158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:48.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f840a800700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55ee86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 -- 172.21.15.50:0/3802088158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 msgr2=0x557d55ee86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55ee86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 -- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 msgr2=0x557d55e885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.157+0000 7f8409400700 1 -- 172.21.15.50:0/3802088158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d55ee8c10 con 0x557d55e71040 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.158+0000 7f8409400700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e88b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009750 tx=0x7f83f8012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b373fe983af5057 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.158+0000 7f83ebe00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.158+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8004a30 con 0x557d55e71040 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.158+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84040c1030 con 0x557d55e71040 2024-05-10T21:13:48.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.158+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f83f8004bd0 con 0x557d55e71040 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83f8004dd0 con 0x557d55e71040 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f841fcd9b80 10 client.16203 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d55eeff70 con 0x557d55e71040 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84040c1030 con 0x557d55e71040 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f83f8028030 con 0x557d55e71040 2024-05-10T21:13:48.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.159+0000 7f841fcd9b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f841fcd9b80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f841fcd9b80 20 client.16203 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55ef0660 0x557d55ef2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f841fcd9b80 4 client.16203 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d55d92400 con 0x557d55ef0660 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.160+0000 7f8409e00700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55ef0660 0x557d55ef2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.161+0000 7f83ebe00700 10 client.16203 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:48.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.161+0000 7f8409e00700 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55ef0660 0x557d55ef2a80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5ca3cf709f52b server_cookie=77925f1a76fbd00e in_seq=0 out_seq=0 2024-05-10T21:13:48.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.161+0000 7f83ebe00700 1 -- 172.21.15.50:0/3802088158 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d55d92400 con 0x557d55ef0660 2024-05-10T21:13:48.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.161+0000 7f83ebe00700 10 client.16203 handle_command_reply: tid=0 2024-05-10T21:13:48.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.161+0000 7f841fcd9b80 1 client.16203 shutdown 2024-05-10T21:13:48.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 msgr2=0x557d55e88b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e88b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009750 tx=0x7f83f8012a30 comp rx=0 tx=0).stop 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 shutdown_connections 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x557d55ef0660 0x557d55ef2a80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d55e6b200 0x557d55e885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d55e75450 0x557d55ee86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 --2- 172.21.15.50:0/3802088158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557d55e71040 0x557d55e88b20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f83ebe00700 0 client.16203 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 >> 172.21.15.50:0/3802088158 conn(0x557d55d29f70 msgr2=0x557d55e74280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 shutdown_connections 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.162+0000 7f841fcd9b80 1 -- 172.21.15.50:0/3802088158 wait complete. 2024-05-10T21:13:48.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.163+0000 7f841fcd9b80 20 client.16203 trim_cache size 0 max 16384 2024-05-10T21:13:48.161 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:13:48.161 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:13:48.161 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:13:48.161 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 msgr2=0x7f83f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 0x7f83f00b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83fc011550 tx=0x7f83fc009260 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 msgr2=0x7f840c1a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 0x7f840c1a18b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f840400b6c0 tx=0x7f840400cc30 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 shutdown_connections 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f83f00b2880 0x7f83f00b4d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f840c110c00 0x7f840c1a18b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c075b00 0x7f840c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 --2- 172.21.15.50:0/147045370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f840c076450 0x7f840c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 >> 172.21.15.50:0/147045370 conn(0x7f840c100040 msgr2=0x7f840c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:48.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 shutdown_connections 2024-05-10T21:13:48.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:48.164+0000 7f8411200700 1 -- 172.21.15.50:0/147045370 wait complete. 2024-05-10T21:13:48.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:48 smithi092 ceph-mon[34415]: pgmap v609: 97 pgs: 97 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-10T21:13:48.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:48 smithi142 ceph-mon[34846]: pgmap v609: 97 pgs: 97 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-10T21:13:49.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:13:49.158 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:13:49.669 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:51.126 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:50 smithi142 ceph-mon[34846]: pgmap v610: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 265 op/s 2024-05-10T21:13:51.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:50 smithi050 ceph-mon[29546]: pgmap v610: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 265 op/s 2024-05-10T21:13:51.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:50 smithi092 ceph-mon[34415]: pgmap v610: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 265 op/s 2024-05-10T21:13:51.398 INFO:tasks.workunit.client.0.smithi050.stdout: 68 145705 86 94285 143 41861 2 2024-05-10T21:13:53.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:52 smithi142 ceph-mon[34846]: pgmap v611: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-05-10T21:13:53.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.151+0000 7fb623e00700 1 -- 172.21.15.50:0/3877413457 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ef0c4cf590 con 0x55ef0c6369e0 2024-05-10T21:13:53.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.151+0000 7fb623e00700 10 client.26324 handle_command_reply: tid=0 2024-05-10T21:13:53.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.151+0000 7fb65742fb80 1 client.26324 shutdown 2024-05-10T21:13:53.149 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:52 smithi050 ceph-mon[29546]: pgmap v611: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-05-10T21:13:53.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.151+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 msgr2=0x55ef0c62f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:53.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.151+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c62f240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb63400a200 tx=0x7fb63400cd30 comp rx=0 tx=0).stop 2024-05-10T21:13:53.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 shutdown_connections 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55ef0c6369e0 0x55ef0c638e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef0c5cfdb0 0x55ef0c629100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ef0c421fd0 0x55ef0c62ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 --2- 172.21.15.50:0/3877413457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef0c5a49b0 0x55ef0c62f240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 >> 172.21.15.50:0/3877413457 conn(0x55ef0c467100 msgr2=0x55ef0c594d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb623e00700 0 client.26324 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 shutdown_connections 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 1 -- 172.21.15.50:0/3877413457 wait complete. 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.152+0000 7fb65742fb80 20 client.26324 trim_cache size 0 max 16384 2024-05-10T21:13:53.151 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 32, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 14, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "f0125e47": { 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 14, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.0, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 85.2, 2024-05-10T21:13:53.152 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:13:53.153 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:13:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.153+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 msgr2=0x7fb6280b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.153+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 0x7fb6280b4f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb63c014c70 tx=0x7fb63c014bc0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.153+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 msgr2=0x7fb6401a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.153+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 0x7fb6401a1460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00be00 tx=0x7fb63c009ed0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 shutdown_connections 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb6280b2a50 0x7fb6280b4f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64010ae90 0x7fb6401a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb640119550 0x7fb6401a1460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 --2- 172.21.15.50:0/488009898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb640116bf0 0x7fb6401a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 >> 172.21.15.50:0/488009898 conn(0x7fb64006f990 msgr2=0x7fb640117000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 shutdown_connections 2024-05-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:53.154+0000 7fb648800700 1 -- 172.21.15.50:0/488009898 wait complete. 2024-05-10T21:13:53.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:52 smithi092 ceph-mon[34415]: pgmap v611: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2024-05-10T21:13:54.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f0125e47', quiesce took 26.8 seconds. Will release after: 25.15 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 -- 172.21.15.50:0/1046153984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 msgr2=0x7f238c0998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 --2- 172.21.15.50:0/1046153984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c0998d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2388009900 tx=0x7f238802a410 comp rx=0 tx=0).stop 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 -- 172.21.15.50:0/1046153984 shutdown_connections 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 --2- 172.21.15.50:0/1046153984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c099e10 0x7f238c09a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 --2- 172.21.15.50:0/1046153984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c09eb10 0x7f238c0a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 --2- 172.21.15.50:0/1046153984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c0998d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.305+0000 7f2399400640 1 -- 172.21.15.50:0/1046153984 >> 172.21.15.50:0/1046153984 conn(0x7f238c093de0 msgr2=0x7f238c096270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.306+0000 7f2399400640 1 -- 172.21.15.50:0/1046153984 shutdown_connections 2024-05-10T21:13:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.306+0000 7f2399400640 1 -- 172.21.15.50:0/1046153984 wait complete. 2024-05-10T21:13:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 Processor -- start 2024-05-10T21:13:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 -- start start 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c12f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c099e10 0x7f238c12f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0a4700 con 0x7f238c099e10 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0a4580 con 0x7f238c098bd0 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.307+0000 7f2399400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0a4880 con 0x7f238c09eb10 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.308+0000 7f2393e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c12f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.308+0000 7f2393400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c099e10 0x7f238c12f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.308+0000 7f2398a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.308+0000 7f2398a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56724/0 (socket says 172.21.15.50:56724) 2024-05-10T21:13:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.308+0000 7f2398a00640 1 -- 172.21.15.50:0/3366101431 learned_addr learned my addr 172.21.15.50:0/3366101431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 -- 172.21.15.50:0/3366101431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 msgr2=0x7f238c12f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c12f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 -- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c099e10 msgr2=0x7f238c12f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c099e10 0x7f238c12f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 -- 172.21.15.50:0/3366101431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c136e80 con 0x7f238c09eb10 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2398a00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2394065d90 tx=0x7f239407b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.309+0000 7f2390c00640 1 -- 172.21.15.50:0/3366101431 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394083070 con 0x7f238c09eb10 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.310+0000 7f2390c00640 1 -- 172.21.15.50:0/3366101431 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394069810 con 0x7f238c09eb10 2024-05-10T21:13:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.310+0000 7f2390c00640 1 -- 172.21.15.50:0/3366101431 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239407f690 con 0x7f238c09eb10 2024-05-10T21:13:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.310+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c137170 con 0x7f238c09eb10 2024-05-10T21:13:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.310+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c137640 con 0x7f238c09eb10 2024-05-10T21:13:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.314+0000 7f23a1a25740 1 Processor -- start 2024-05-10T21:13:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.315+0000 7f2390c00640 1 -- 172.21.15.50:0/3366101431 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f239407f830 con 0x7f238c09eb10 2024-05-10T21:13:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.315+0000 7f23a1a25740 1 -- start start 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 0x55afdd79d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afdd701550 con 0x55afdd3d22f0 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55afdd6a6bc0 con 0x55afdd3da4d0 2024-05-10T21:13:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f23a1a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55afdd6fd510 con 0x55afdd3fb1b0 2024-05-10T21:13:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f2393400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f2393400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56736/0 (socket says 172.21.15.50:56736) 2024-05-10T21:13:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f2393400640 1 -- 172.21.15.50:0/1686124846 learned_addr learned my addr 172.21.15.50:0/1686124846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.316+0000 7f2393e00640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 0x55afdd79d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 -- 172.21.15.50:0/1686124846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 msgr2=0x55afdd7a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 -- 172.21.15.50:0/1686124846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 msgr2=0x55afdd79d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 0x55afdd79d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.317+0000 7f2393400640 1 -- 172.21.15.50:0/1686124846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdd413210 con 0x55afdd3fb1b0 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.318+0000 7f2393e00640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 0x55afdd79d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.318+0000 7f2390c00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 0x7f237c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.318+0000 7f2390c00640 1 -- 172.21.15.50:0/3366101431 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f239414f2f0 con 0x7f238c09eb10 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.318+0000 7f2393400640 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23800145b0 tx=0x7f238000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a599d5819a105d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.318+0000 7f2393e00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 0x7f237c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.319+0000 7f2385200640 1 -- 172.21.15.50:0/1686124846 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238001a070 con 0x55afdd3fb1b0 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.319+0000 7f2385200640 1 -- 172.21.15.50:0/1686124846 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.319+0000 7f2385200640 1 -- 172.21.15.50:0/1686124846 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800107f0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 -- 172.21.15.50:0/1686124846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 msgr2=0x55afdd79d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23800145b0 tx=0x7f238000b960 comp rx=0 tx=0).stop 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f2393e00640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 0x7f237c0b4d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f238802aa00 tx=0x7f23880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 -- 172.21.15.50:0/1686124846 shutdown_connections 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3d22f0 0x55afdd79d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 --2- 172.21.15.50:0/1686124846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3fb1b0 0x55afdd79d2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 -- 172.21.15.50:0/1686124846 >> 172.21.15.50:0/1686124846 conn(0x55afdd3ef300 msgr2=0x55afdd798c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 -- 172.21.15.50:0/1686124846 shutdown_connections 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.320+0000 7f23a1a25740 1 -- 172.21.15.50:0/1686124846 wait complete. 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.321+0000 7f23a1a25740 1 -- start start 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.321+0000 7f23a1a25740 1 Processor -- start 2024-05-10T21:13:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3da4d0 0x55afdd7a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 0x55afdd7ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afdd7a51c0 con 0x55afdd3da4d0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55afdd6ba660 con 0x55afdd3fb1b0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55afdd7a5340 con 0x55afdd3d22f0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f23a1a25740 10 client.0 _ll_register_callbacks cb 0x55afdd684400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f2393e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f2398a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 0x55afdd7ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f2393e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7a6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56752/0 (socket says 172.21.15.50:56752) 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.322+0000 7f2393e00640 1 -- 172.21.15.50:0/2552930610 learned_addr learned my addr 172.21.15.50:0/2552930610 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2393400640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3da4d0 0x55afdd7a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 -- 172.21.15.50:0/2552930610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 msgr2=0x55afdd7a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 -- 172.21.15.50:0/2552930610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3da4d0 msgr2=0x55afdd7a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3da4d0 0x55afdd7a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 -- 172.21.15.50:0/2552930610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdd4df620 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2398a00640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 0x55afdd7ae370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2394052220 tx=0x7f2394116f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25215f6e4f7b892e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.323+0000 7f2379600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394116430 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2394137bd0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23800035f0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2394117020 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f23a1a25740 10 client.26354 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afdd7b1370 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.324+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f2394115b80 con 0x55afdd3fb1b0 2024-05-10T21:13:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 10 client.26354 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 10 client.26354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 20 client.26354 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd3b9d80 0x55afdd7b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 4 client.26354 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afdd70f160 con 0x55afdd3b9d80 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.325+0000 7f2393e00640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd3b9d80 0x55afdd7b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.326+0000 7f2393e00640 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd3b9d80 0x55afdd7b4a60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761fc7a0e858d8f3 server_cookie=32ccbce9f3db110c in_seq=0 out_seq=0 2024-05-10T21:13:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.326+0000 7f2379600640 10 client.26354 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f2379600640 1 -- 172.21.15.50:0/2552930610 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afdd70f160 con 0x55afdd3b9d80 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f2379600640 10 client.26354 handle_command_reply: tid=0 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 client.26354 shutdown 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 msgr2=0x55afdd7ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 0x55afdd7ae370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2394052220 tx=0x7f2394116f00 comp rx=0 tx=0).stop 2024-05-10T21:13:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 shutdown_connections 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd3b9d80 0x55afdd7b4a60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3fb1b0 0x55afdd7ae370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3da4d0 0x55afdd7a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 --2- 172.21.15.50:0/2552930610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f2379600640 0 client.26354 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 >> 172.21.15.50:0/2552930610 conn(0x55afdd3ef300 msgr2=0x55afdd797c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 shutdown_connections 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.327+0000 7f23a1a25740 1 -- 172.21.15.50:0/2552930610 wait complete. 2024-05-10T21:13:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.328+0000 7f23a1a25740 20 client.26354 trim_cache size 0 max 16384 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.348+0000 7f23a1a25740 1 Processor -- start 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.348+0000 7f23a1a25740 1 -- start start 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 0x55afdd7a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 0x55afdd7ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd79c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f238c142aa0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1402e0 con 0x55afdd3d22f0 2024-05-10T21:13:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f23a1a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f238c13e130 con 0x55afdd3da4d0 2024-05-10T21:13:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f2393e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 0x55afdd7a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f2393e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 0x55afdd7a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43084/0 (socket says 172.21.15.50:43084) 2024-05-10T21:13:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f2393e00640 1 -- 172.21.15.50:0/93090951 learned_addr learned my addr 172.21.15.50:0/93090951 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f2398a00640 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd79c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.349+0000 7f2393400640 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 0x55afdd7ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 -- 172.21.15.50:0/93090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 msgr2=0x55afdd7a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 0x55afdd7a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 -- 172.21.15.50:0/93090951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 msgr2=0x55afdd79c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd79c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 -- 172.21.15.50:0/93090951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdd7a0e60 con 0x55afdd3da4d0 2024-05-10T21:13:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.350+0000 7f2393400640 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 0x55afdd7ae290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55afdd799970 tx=0x7f2380005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8221e9ccf873cd49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.351+0000 7f237be00640 1 -- 172.21.15.50:0/93090951 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238001a040 con 0x55afdd3da4d0 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.351+0000 7f237be00640 1 -- 172.21.15.50:0/93090951 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238000b770 con 0x55afdd3da4d0 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.351+0000 7f237be00640 1 -- 172.21.15.50:0/93090951 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380003d00 con 0x55afdd3da4d0 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- 172.21.15.50:0/93090951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 msgr2=0x55afdd7ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 0x55afdd7ae290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55afdd799970 tx=0x7f2380005bb0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- 172.21.15.50:0/93090951 shutdown_connections 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3da4d0 0x55afdd7ae290 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd79c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 --2- 172.21.15.50:0/93090951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3d22f0 0x55afdd7a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- 172.21.15.50:0/93090951 >> 172.21.15.50:0/93090951 conn(0x55afdd3ef300 msgr2=0x55afdd79ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- 172.21.15.50:0/93090951 shutdown_connections 2024-05-10T21:13:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- 172.21.15.50:0/93090951 wait complete. 2024-05-10T21:13:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.352+0000 7f23a1a25740 1 -- start start 2024-05-10T21:13:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 Processor -- start 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f238c13e7a0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f238c13fc10 con 0x55afdd3da4d0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f238c13ee70 con 0x55afdd3d22f0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.353+0000 7f23a1a25740 10 client.0 _ll_register_callbacks cb 0x7f2394069e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36708/0 (socket says 172.21.15.50:36708) 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 -- 172.21.15.50:0/3200105236 learned_addr learned my addr 172.21.15.50:0/3200105236 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2393e00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2393400640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 -- 172.21.15.50:0/3200105236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 msgr2=0x55afdd7ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 -- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 msgr2=0x55afdd7adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 -- 172.21.15.50:0/3200105236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdd81b800 con 0x55afdd3fb1b0 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2398a00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f239412d040 tx=0x7f2394064250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d41e66dadf84c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.354+0000 7f2379600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:13:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23941120b0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f239406c170 con 0x55afdd3fb1b0 2024-05-10T21:13:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238000b770 con 0x55afdd3fb1b0 2024-05-10T21:13:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2394116430 con 0x55afdd3fb1b0 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f23a1a25740 10 client.16221 fetch_fsmap learned FSMap version 32 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.355+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afdd8252d0 con 0x55afdd3fb1b0 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.356+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238000b770 con 0x55afdd3fb1b0 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f2394112290 con 0x55afdd3fb1b0 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 10 client.16221 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 20 client.16221 populate_metadata read hostname 'smithi050' 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd38f1c0 0x55afdd827b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 4 client.16221 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afdd708fd0 con 0x55afdd38f1c0 2024-05-10T21:13:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.357+0000 7f2393e00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd38f1c0 0x55afdd827b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.358+0000 7f2393e00640 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd38f1c0 0x55afdd827b60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd688a59bf06db7e server_cookie=70d97a2f68655c3b in_seq=0 out_seq=0 2024-05-10T21:13:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:13:54.358+0000 7f2379600640 10 client.16221 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: pgmap v612: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:54 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: pgmap v612: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:54 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: pgmap v612: 97 pgs: 97 active+clean; 284 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-05-10T21:13:55.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:13:55.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:54 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T21:13:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.797+0000 7f7f53000700 1 -- 172.21.15.50:0/2975978190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c108f20 msgr2=0x7f7f4c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:55.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.797+0000 7f7f53000700 1 --2- 172.21.15.50:0/2975978190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c108f20 0x7f7f4c109330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098f0 tx=0x7f7f38009c00 comp rx=0 tx=0).stop 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.797+0000 7f7f53000700 1 -- 172.21.15.50:0/2975978190 shutdown_connections 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 --2- 172.21.15.50:0/2975978190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c109870 0x7f7f4c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 --2- 172.21.15.50:0/2975978190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 --2- 172.21.15.50:0/2975978190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c108f20 0x7f7f4c109330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 -- 172.21.15.50:0/2975978190 >> 172.21.15.50:0/2975978190 conn(0x7f7f4c100040 msgr2=0x7f7f4c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 -- 172.21.15.50:0/2975978190 shutdown_connections 2024-05-10T21:13:55.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.798+0000 7f7f53000700 1 -- 172.21.15.50:0/2975978190 wait complete. 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.799+0000 7f7f53000700 1 Processor -- start 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.799+0000 7f7f53000700 1 -- start start 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 0x7f7f4c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1144d0 con 0x7f7f4c108f20 2024-05-10T21:13:55.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c114350 con 0x7f7f4c109870 2024-05-10T21:13:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f53000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c114650 con 0x7f7f4c1045f0 2024-05-10T21:13:55.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36730/0 (socket says 172.21.15.50:36730) 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56782/0 (socket says 172.21.15.50:56782) 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f4b400700 1 -- 172.21.15.50:0/22884998 learned_addr learned my addr 172.21.15.50:0/22884998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.800+0000 7f7f51000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 0x7f7f4c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 -- 172.21.15.50:0/22884998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 msgr2=0x7f7f4c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 0x7f7f4c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 -- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 msgr2=0x7f7f4c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 -- 172.21.15.50:0/22884998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c1a20d0 con 0x7f7f4c1045f0 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4b400700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f51000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 0x7f7f4c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:13:55.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.801+0000 7f7f4be00700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00e9f0 tx=0x7f7f3c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.802+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00cb70 con 0x7f7f4c1045f0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.802+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c00ee90 con 0x7f7f4c1045f0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.802+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c003d40 con 0x7f7f4c1045f0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.802+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c1a23c0 con 0x7f7f4c1045f0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.802+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a9ce0 con 0x7f7f4c1045f0 2024-05-10T21:13:55.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.803+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c0512c0 con 0x7f7f4c1045f0 2024-05-10T21:13:55.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.806+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f3c003ee0 con 0x7f7f4c1045f0 2024-05-10T21:13:55.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.807+0000 7f7f48c00700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 0x7f7f300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:13:55.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.807+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7f3c0e2480 con 0x7f7f4c1045f0 2024-05-10T21:13:55.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.808+0000 7f7f4b400700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 0x7f7f300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:13:55.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.808+0000 7f7f4b400700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 0x7f7f300b4e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005f40 tx=0x7f7f38019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:13:55.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:55.809+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f3c054df0 con 0x7f7f4c1045f0 2024-05-10T21:13:56.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:55 smithi050 ceph-mon[29546]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-10T21:13:56.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:55 smithi050 ceph-mon[29546]: Unable to set osd_memory_target on smithi142 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-10T21:13:56.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:13:56.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:13:56.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:13:56.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.182+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4c19e560 con 0x7f7f4c1045f0 2024-05-10T21:13:56.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.183+0000 7f7f48c00700 1 -- 172.21.15.50:0/22884998 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11856 (secure 0 0 0) 0x7f7f3c0aa170 con 0x7f7f4c1045f0 2024-05-10T21:13:56.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:13:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:13:39.382028+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24502,24520,24539]},"id":1}]} 2024-05-10T21:13:56.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.186+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 msgr2=0x7f7f300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:56.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 0x7f7f300b4e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005f40 tx=0x7f7f38019040 comp rx=0 tx=0).stop 2024-05-10T21:13:56.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 msgr2=0x7f7f4c19d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:13:56.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00e9f0 tx=0x7f7f3c00ed00 comp rx=0 tx=0).stop 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 shutdown_connections 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f300b2900 0x7f7f300b4e10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c109870 0x7f7f4c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4c1045f0 0x7f7f4c19d200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 --2- 172.21.15.50:0/22884998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f4c108f20 0x7f7f4c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 >> 172.21.15.50:0/22884998 conn(0x7f7f4c100040 msgr2=0x7f7f4c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 shutdown_connections 2024-05-10T21:13:56.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:13:56.187+0000 7f7f53000700 1 -- 172.21.15.50:0/22884998 wait complete. 2024-05-10T21:13:56.186 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 32 2024-05-10T21:13:56.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:55 smithi092 ceph-mon[34415]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-10T21:13:56.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:55 smithi092 ceph-mon[34415]: Unable to set osd_memory_target on smithi142 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-10T21:13:56.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:13:56.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:13:56.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:13:56.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:55 smithi142 ceph-mon[34846]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-10T21:13:56.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:55 smithi142 ceph-mon[34846]: Unable to set osd_memory_target on smithi142 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-10T21:13:56.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:13:56.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:13:56.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:13:57.025 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:56 smithi050 ceph-mon[29546]: pgmap v613: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-05-10T21:13:57.025 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:56 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/22884998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:57.090 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:13:57.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:56 smithi092 ceph-mon[34415]: pgmap v613: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-05-10T21:13:57.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:56 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/22884998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:57.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:56 smithi142 ceph-mon[34846]: pgmap v613: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-05-10T21:13:57.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:56 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/22884998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:13:57.578 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:13:59.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:13:58 smithi050 ceph-mon[29546]: pgmap v614: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-05-10T21:13:59.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:13:58 smithi092 ceph-mon[34415]: pgmap v614: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-05-10T21:13:59.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:13:58 smithi142 ceph-mon[34846]: pgmap v614: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-05-10T21:14:00.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:00 smithi050 ceph-mon[29546]: pgmap v615: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T21:14:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:00 smithi092 ceph-mon[34415]: pgmap v615: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T21:14:01.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:00 smithi142 ceph-mon[34846]: pgmap v615: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T21:14:01.398 INFO:tasks.workunit.client.0.smithi050.stdout: 68 81144 0 52175 0 24384 1 2024-05-10T21:14:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 -- 172.21.15.50:0/74448471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 msgr2=0x7fe554104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 --2- 172.21.15.50:0/74448471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554104810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe5440099c0 tx=0x7fe544009cd0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 -- 172.21.15.50:0/74448471 shutdown_connections 2024-05-10T21:14:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 --2- 172.21.15.50:0/74448471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554109680 0x7fe554110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 --2- 172.21.15.50:0/74448471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554104810 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 --2- 172.21.15.50:0/74448471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554108d30 0x7fe554109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 -- 172.21.15.50:0/74448471 >> 172.21.15.50:0/74448471 conn(0x7fe554079d10 msgr2=0x7fe55407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 -- 172.21.15.50:0/74448471 shutdown_connections 2024-05-10T21:14:02.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.639+0000 7fe559a00700 1 -- 172.21.15.50:0/74448471 wait complete. 2024-05-10T21:14:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.640+0000 7fe559a00700 1 Processor -- start 2024-05-10T21:14:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 -- start start 2024-05-10T21:14:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554108d30 0x7fe554076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 0x7fe554076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe554114300 con 0x7fe554104400 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe554114180 con 0x7fe554108d30 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe554114480 con 0x7fe554109680 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe553e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe553e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36752/0 (socket says 172.21.15.50:36752) 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe553e00700 1 -- 172.21.15.50:0/2445674597 learned_addr learned my addr 172.21.15.50:0/2445674597 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.641+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 0x7fe554076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe553400700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554108d30 0x7fe554076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 -- 172.21.15.50:0/2445674597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554108d30 msgr2=0x7fe554076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554108d30 0x7fe554076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 -- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 msgr2=0x7fe554075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 -- 172.21.15.50:0/2445674597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5541a9070 con 0x7fe554109680 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.642+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 0x7fe554076540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe54c007f10 tx=0x7fe54c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.643+0000 7fe550c00700 1 -- 172.21.15.50:0/2445674597 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004db0 con 0x7fe554109680 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.643+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5541a9360 con 0x7fe554109680 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.643+0000 7fe550c00700 1 -- 172.21.15.50:0/2445674597 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c003a00 con 0x7fe554109680 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.643+0000 7fe550c00700 1 -- 172.21.15.50:0/2445674597 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004580 con 0x7fe554109680 2024-05-10T21:14:02.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.643+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5541a98a0 con 0x7fe554109680 2024-05-10T21:14:02.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.647+0000 7fe550c00700 1 -- 172.21.15.50:0/2445674597 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe54c01c020 con 0x7fe554109680 2024-05-10T21:14:02.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.647+0000 7fe569bccb80 1 Processor -- start 2024-05-10T21:14:02.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 -- start start 2024-05-10T21:14:02.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00a5de40 0x55ce00bdbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ce00afda20 con 0x55ce00a5de40 2024-05-10T21:14:02.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ce00a9b760 con 0x55ce00bdc280 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.648+0000 7fe569bccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce00af99e0 con 0x55ce00bdc690 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00a5de40 0x55ce00bdbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43136/0 (socket says 172.21.15.50:43136) 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 -- 172.21.15.50:0/3616269119 learned_addr learned my addr 172.21.15.50:0/3616269119 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:02.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe550c00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 0x7fe5380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553400700 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe550c00700 1 -- 172.21.15.50:0/2445674597 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe54c0e1860 con 0x7fe554109680 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 -- 172.21.15.50:0/3616269119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 msgr2=0x55ce00be7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 -- 172.21.15.50:0/3616269119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00a5de40 msgr2=0x55ce00bdbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00a5de40 0x55ce00bdbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe553e00700 1 -- 172.21.15.50:0/3616269119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce00a4a8d0 con 0x55ce00bdc280 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.649+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 0x7fe5380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.650+0000 7fe553e00700 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe544003730 tx=0x7fe544005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ca77b593cb4093 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.650+0000 7fe549600700 1 -- 172.21.15.50:0/3616269119 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544027070 con 0x55ce00bdc280 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.650+0000 7fe549600700 1 -- 172.21.15.50:0/3616269119 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5440037c0 con 0x55ce00bdc280 2024-05-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.650+0000 7fe549600700 1 -- 172.21.15.50:0/3616269119 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400b9f0 con 0x55ce00bdc280 2024-05-10T21:14:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.650+0000 7fe559000700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 0x7fe5380b4e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0a60c0 tx=0x7fe54c004f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.651+0000 7fe569bccb80 1 -- 172.21.15.50:0/3616269119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 msgr2=0x55ce00be58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.651+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe544003730 tx=0x7fe544005040 comp rx=0 tx=0).stop 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 -- 172.21.15.50:0/3616269119 shutdown_connections 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00a5de40 0x55ce00bdbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3616269119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 -- 172.21.15.50:0/3616269119 >> 172.21.15.50:0/3616269119 conn(0x55ce00a9ca80 msgr2=0x55ce00bd6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 -- 172.21.15.50:0/3616269119 shutdown_connections 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 -- 172.21.15.50:0/3616269119 wait complete. 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.652+0000 7fe569bccb80 1 -- start start 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 Processor -- start 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00be5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ce00be8740 con 0x55ce00bdc690 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ce00a63b20 con 0x55ce00bdc280 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce00be88c0 con 0x55ce00a5de40 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.653+0000 7fe569bccb80 10 client.0 _ll_register_callbacks cb 0x55ce00a91380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56810/0 (socket says 172.21.15.50:56810) 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 -- 172.21.15.50:0/3160804135 learned_addr learned my addr 172.21.15.50:0/3160804135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00be5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553e00700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 -- 172.21.15.50:0/3160804135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 msgr2=0x55ce00be58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 -- 172.21.15.50:0/3160804135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 msgr2=0x55ce00be5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00be5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.654+0000 7fe553400700 1 -- 172.21.15.50:0/3160804135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce00ac3420 con 0x55ce00a5de40 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe553400700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe54000f430 tx=0x7fe540015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b741fa09bc24e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe53e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54001c040 con 0x55ce00a5de40 2024-05-10T21:14:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe54000fd10 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5440037c0 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe540004050 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe569bccb80 10 client.26329 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.655+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce00bf3450 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5440037c0 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe540003b80 con 0x55ce00a5de40 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 10 client.26329 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 10 client.26329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 20 client.26329 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00bf59e0 0x55ce00bf7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 4 client.26329 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce00ba8070 con 0x55ce00bf59e0 2024-05-10T21:14:02.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.656+0000 7fe553e00700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00bf59e0 0x55ce00bf7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.657+0000 7fe553e00700 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00bf59e0 0x55ce00bf7e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fd86ae57343bfc0 server_cookie=6e06eb8a5889aea4 in_seq=0 out_seq=0 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.657+0000 7fe53e000700 10 client.26329 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.658+0000 7fe53e000700 1 -- 172.21.15.50:0/3160804135 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce00ba8070 con 0x55ce00bf59e0 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.658+0000 7fe53e000700 10 client.26329 handle_command_reply: tid=0 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.658+0000 7fe569bccb80 1 client.26329 shutdown 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 msgr2=0x55ce00be53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe54000f430 tx=0x7fe540015040 comp rx=0 tx=0).stop 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 shutdown_connections 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00bf59e0 0x55ce00bf7e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00be5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be53a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3160804135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe53e000700 0 client.26329 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 >> 172.21.15.50:0/3160804135 conn(0x55ce00a9ca80 msgr2=0x55ce00bda700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 shutdown_connections 2024-05-10T21:14:02.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 1 -- 172.21.15.50:0/3160804135 wait complete. 2024-05-10T21:14:02.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.659+0000 7fe569bccb80 20 client.26329 trim_cache size 0 max 16384 2024-05-10T21:14:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.691+0000 7fe569bccb80 1 Processor -- start 2024-05-10T21:14:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.691+0000 7fe569bccb80 1 -- start start 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.692+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.692+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe569bccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541b34b0 con 0x55ce00bdc280 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe569bccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541b0cb0 con 0x55ce00a5de40 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe569bccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541aeb20 con 0x55ce00bdc690 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43154/0 (socket says 172.21.15.50:43154) 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553400700 1 -- 172.21.15.50:0/3051816243 learned_addr learned my addr 172.21.15.50:0/3051816243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36770/0 (socket says 172.21.15.50:36770) 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.693+0000 7fe553400700 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 -- 172.21.15.50:0/3051816243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 msgr2=0x55ce00be5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 -- 172.21.15.50:0/3051816243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 msgr2=0x55ce00be4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 -- 172.21.15.50:0/3051816243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce00bce640 con 0x55ce00bdc280 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553400700 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe553e00700 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe544006010 tx=0x7fe544005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1486213591c450ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe548c00700 1 -- 172.21.15.50:0/3051816243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544027040 con 0x55ce00bdc280 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe548c00700 1 -- 172.21.15.50:0/3051816243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe544032070 con 0x55ce00bdc280 2024-05-10T21:14:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.694+0000 7fe548c00700 1 -- 172.21.15.50:0/3051816243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400bb10 con 0x55ce00bdc280 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 -- 172.21.15.50:0/3051816243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 msgr2=0x55ce00be7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe544006010 tx=0x7fe544005a90 comp rx=0 tx=0).stop 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 -- 172.21.15.50:0/3051816243 shutdown_connections 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00a5de40 0x55ce00be4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00bdc690 0x55ce00be5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.695+0000 7fe569bccb80 1 --2- 172.21.15.50:0/3051816243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc280 0x55ce00be7480 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.696+0000 7fe569bccb80 1 -- 172.21.15.50:0/3051816243 >> 172.21.15.50:0/3051816243 conn(0x55ce00a9ca80 msgr2=0x55ce00bd5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.696+0000 7fe569bccb80 1 -- 172.21.15.50:0/3051816243 shutdown_connections 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.696+0000 7fe569bccb80 1 -- 172.21.15.50:0/3051816243 wait complete. 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.696+0000 7fe569bccb80 1 -- start start 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.696+0000 7fe569bccb80 1 Processor -- start 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00c5e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541af170 con 0x55ce00bdc690 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541b05e0 con 0x55ce00bdc280 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541af840 con 0x55ce00a5de40 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe569bccb80 10 client.0 _ll_register_callbacks cb 0x55ce00a91380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe553400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00c5e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe553400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe553400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56832/0 (socket says 172.21.15.50:56832) 2024-05-10T21:14:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe553e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.697+0000 7fe553400700 1 -- 172.21.15.50:0/2278136929 learned_addr learned my addr 172.21.15.50:0/2278136929 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 -- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 msgr2=0x55ce00be8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 -- 172.21.15.50:0/2278136929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 msgr2=0x55ce00c5e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00c5e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 -- 172.21.15.50:0/2278136929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce00c5e6c0 con 0x55ce00bdc280 2024-05-10T21:14:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553400700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe553e00700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be92f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe544025040 tx=0x7fe5440037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d45ed071e8d342 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.698+0000 7fe53e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544027040 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe544032070 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe544005170 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5440053b0 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe569bccb80 10 client.26384 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ce00c5f880 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.699+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe544032070 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe544021030 con 0x55ce00bdc280 2024-05-10T21:14:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 10 client.26384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 20 client.26384 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00c5fa50 0x55ce00c61e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 4 client.26384 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce00b0b400 con 0x55ce00c5fa50 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.700+0000 7fe553400700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00c5fa50 0x55ce00c61e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe553400700 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00c5fa50 0x55ce00c61e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef4a8641ce425cab server_cookie=ff0e79a0227021a8 in_seq=0 out_seq=0 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe53e000700 10 client.26384 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe53e000700 1 -- 172.21.15.50:0/2278136929 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce00b0b400 con 0x55ce00c5fa50 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe53e000700 10 client.26384 handle_command_reply: tid=0 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe569bccb80 1 client.26384 shutdown 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 msgr2=0x55ce00be92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be92f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe544025040 tx=0x7fe5440037c0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 shutdown_connections 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.701+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55ce00c5fa50 0x55ce00c61e70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ce00bdc690 0x55ce00c5e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce00a5de40 0x55ce00be8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe53e000700 0 client.26384 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 --2- 172.21.15.50:0/2278136929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ce00bdc280 0x55ce00be92f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 >> 172.21.15.50:0/2278136929 conn(0x55ce00a9ca80 msgr2=0x55ce00be7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 shutdown_connections 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 1 -- 172.21.15.50:0/2278136929 wait complete. 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe569bccb80 20 client.26384 trim_cache size 0 max 16384 2024-05-10T21:14:02.702 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:14:02.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.702+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 msgr2=0x7fe5380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 0x7fe5380b4e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0a60c0 tx=0x7fe54c004f50 comp rx=0 tx=0).stop 2024-05-10T21:14:02.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 msgr2=0x7fe554076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:02.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 0x7fe554076540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe54c007f10 tx=0x7fe54c00d3b0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 shutdown_connections 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe5380b2900 0x7fe5380b4e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe554109680 0x7fe554076540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe554104400 0x7fe554075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 --2- 172.21.15.50:0/2445674597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe554108d30 0x7fe554076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 >> 172.21.15.50:0/2445674597 conn(0x7fe554079d10 msgr2=0x7fe554101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 shutdown_connections 2024-05-10T21:14:02.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:02.703+0000 7fe559a00700 1 -- 172.21.15.50:0/2445674597 wait complete. 2024-05-10T21:14:02.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:02 smithi050 ceph-mon[29546]: pgmap v616: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-10T21:14:03.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:02 smithi092 ceph-mon[34415]: pgmap v616: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-10T21:14:03.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:02 smithi142 ceph-mon[34846]: pgmap v616: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-05-10T21:14:03.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:14:04.055 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:14:05.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:04 smithi050 ceph-mon[29546]: pgmap v617: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 337 KiB/s wr, 55 op/s 2024-05-10T21:14:05.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:04 smithi092 ceph-mon[34415]: pgmap v617: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 337 KiB/s wr, 55 op/s 2024-05-10T21:14:05.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:04 smithi142 ceph-mon[34846]: pgmap v617: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 337 KiB/s wr, 55 op/s 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 -- 172.21.15.50:0/2663669780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 msgr2=0x7f0a6c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 --2- 172.21.15.50:0/2663669780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c109140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a580099c0 tx=0x7f0a58009cd0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 -- 172.21.15.50:0/2663669780 shutdown_connections 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 --2- 172.21.15.50:0/2663669780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 --2- 172.21.15.50:0/2663669780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 --2- 172.21.15.50:0/2663669780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c109140 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 -- 172.21.15.50:0/2663669780 >> 172.21.15.50:0/2663669780 conn(0x7f0a6c079d10 msgr2=0x7f0a6c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 -- 172.21.15.50:0/2663669780 shutdown_connections 2024-05-10T21:14:07.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.082+0000 7f0a6b200700 1 -- 172.21.15.50:0/2663669780 wait complete. 2024-05-10T21:14:07.081 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:06 smithi050 ceph-mon[29546]: pgmap v618: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 52 KiB/s wr, 40 op/s 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.083+0000 7f0a6b200700 1 Processor -- start 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.083+0000 7f0a6b200700 1 -- start start 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c114290 con 0x7f0a6c109680 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c114110 con 0x7f0a6c104400 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c114410 con 0x7f0a6c108d30 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6a800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a69400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a6a800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36052/0 (socket says 172.21.15.50:36052) 2024-05-10T21:14:07.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a69400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41894/0 (socket says 172.21.15.50:41894) 2024-05-10T21:14:07.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a69400700 1 -- 172.21.15.50:0/1775673975 learned_addr learned my addr 172.21.15.50:0/1775673975 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:07.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.084+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41644/0 (socket says 172.21.15.50:41644) 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 -- 172.21.15.50:0/1775673975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 msgr2=0x7f0a6c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 -- 172.21.15.50:0/1775673975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 msgr2=0x7f0a6c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 -- 172.21.15.50:0/1775673975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c1a8fc0 con 0x7f0a6c109680 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a69e00700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.085+0000 7f0a6a800700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400efc0 tx=0x7f0a6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.086+0000 7f0a62a00700 1 -- 172.21.15.50:0/1775673975 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a640092e0 con 0x7f0a6c109680 2024-05-10T21:14:07.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.086+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c1a9250 con 0x7f0a6c109680 2024-05-10T21:14:07.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.086+0000 7f0a62a00700 1 -- 172.21.15.50:0/1775673975 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a64004bb0 con 0x7f0a6c109680 2024-05-10T21:14:07.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.086+0000 7f0a62a00700 1 -- 172.21.15.50:0/1775673975 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64004020 con 0x7f0a6c109680 2024-05-10T21:14:07.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.086+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c1a9760 con 0x7f0a6c109680 2024-05-10T21:14:07.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.090+0000 7f0a62a00700 1 -- 172.21.15.50:0/1775673975 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0a6401b030 con 0x7f0a6c109680 2024-05-10T21:14:07.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.091+0000 7f0a7f642b80 1 Processor -- start 2024-05-10T21:14:07.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.091+0000 7f0a7f642b80 1 -- start start 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f0634d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06347700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06348050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f06269a20 con 0x562f06347c40 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562f06207760 con 0x562f0634da80 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f062659e0 con 0x562f061c9e40 2024-05-10T21:14:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a62a00700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a62a00700 1 -- 172.21.15.50:0/1775673975 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a640e1ca0 con 0x7f0a6c109680 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a6a800700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a69400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f0634d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a69400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f0634d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41906/0 (socket says 172.21.15.50:41906) 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a69400700 1 -- 172.21.15.50:0/1367069593 learned_addr learned my addr 172.21.15.50:0/1367069593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06347700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.092+0000 7f0a69400700 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06348050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 -- 172.21.15.50:0/1367069593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 msgr2=0x562f0634d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f0634d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 -- 172.21.15.50:0/1367069593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 msgr2=0x562f06348050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06348050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 -- 172.21.15.50:0/1367069593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f061b68d0 con 0x562f0634da80 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a69e00700 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06347700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0114f0 tx=0x7f0a5c011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a18725bed9ea36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a6a800700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 0x7f0a500b4d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0a640a9020 tx=0x7f0a64009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.093+0000 7f0a57400700 1 -- 172.21.15.50:0/1367069593 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c011310 con 0x562f0634da80 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.094+0000 7f0a57400700 1 -- 172.21.15.50:0/1367069593 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5c0044e0 con 0x562f0634da80 2024-05-10T21:14:07.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.094+0000 7f0a57400700 1 -- 172.21.15.50:0/1367069593 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c01a6e0 con 0x562f0634da80 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.094+0000 7f0a7f642b80 1 -- 172.21.15.50:0/1367069593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 msgr2=0x562f06347700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.094+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06347700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0114f0 tx=0x7f0a5c011800 comp rx=0 tx=0).stop 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 -- 172.21.15.50:0/1367069593 shutdown_connections 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06348050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f0634d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/1367069593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06347700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 -- 172.21.15.50:0/1367069593 >> 172.21.15.50:0/1367069593 conn(0x562f0620ef70 msgr2=0x562f06342400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 -- 172.21.15.50:0/1367069593 shutdown_connections 2024-05-10T21:14:07.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 -- 172.21.15.50:0/1367069593 wait complete. 2024-05-10T21:14:07.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.095+0000 7f0a7f642b80 1 -- start start 2024-05-10T21:14:07.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 Processor -- start 2024-05-10T21:14:07.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f06347c40 0x562f06351690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06351bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f06354490 con 0x562f061c9e40 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562f061cfb20 con 0x562f0634da80 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f06354610 con 0x562f06347c40 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.096+0000 7f0a7f642b80 10 client.0 _ll_register_callbacks cb 0x562f06210c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f06347c40 0x562f06351690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36076/0 (socket says 172.21.15.50:36076) 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 -- 172.21.15.50:0/429593166 learned_addr learned my addr 172.21.15.50:0/429593166 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06351bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 -- 172.21.15.50:0/429593166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f06347c40 msgr2=0x562f06351690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f06347c40 0x562f06351690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 -- 172.21.15.50:0/429593166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 msgr2=0x562f06351bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06351bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.097+0000 7f0a69400700 1 -- 172.21.15.50:0/429593166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f0622f420 con 0x562f061c9e40 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a69400700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009990 tx=0x7f0a58009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1272115eb3dcb563 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a4be00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5802e060 con 0x562f061c9e40 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a5800b790 con 0x562f061c9e40 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a5c0044e0 con 0x562f061c9e40 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a58003e40 con 0x562f061c9e40 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a7f642b80 10 client.16257 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:07.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.098+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f0635f030 con 0x562f061c9e40 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5c0044e0 con 0x562f061c9e40 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f0a5800f8e0 con 0x562f061c9e40 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 20 client.16257 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f06361670 0x562f06363a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 4 client.16257 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f063141c0 con 0x562f06361670 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.099+0000 7f0a69e00700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f06361670 0x562f06363a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.100+0000 7f0a69e00700 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f06361670 0x562f06363a90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7353055551bc09c1 server_cookie=cbc929265837fa27 in_seq=0 out_seq=0 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.100+0000 7f0a4be00700 10 client.16257 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a4be00700 1 -- 172.21.15.50:0/429593166 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f063141c0 con 0x562f06361670 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a4be00700 10 client.16257 handle_command_reply: tid=0 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 client.16257 shutdown 2024-05-10T21:14:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 msgr2=0x562f06351150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009990 tx=0x7f0a58009cd0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 shutdown_connections 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06351bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f06361670 0x562f06363a90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f061c9e40 0x562f06351150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/429593166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f06347c40 0x562f06351690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a4be00700 0 client.16257 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:14:07.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.101+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 >> 172.21.15.50:0/429593166 conn(0x562f0620ef70 msgr2=0x562f063466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.102+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 shutdown_connections 2024-05-10T21:14:07.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.102+0000 7f0a7f642b80 1 -- 172.21.15.50:0/429593166 wait complete. 2024-05-10T21:14:07.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.102+0000 7f0a7f642b80 20 client.16257 trim_cache size 0 max 16384 2024-05-10T21:14:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 Processor -- start 2024-05-10T21:14:07.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 -- start start 2024-05-10T21:14:07.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0634da80 0x562f06351a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1b3490 con 0x562f06347c40 2024-05-10T21:14:07.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1b0c90 con 0x562f061c9e40 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.119+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1aeb40 con 0x562f0634da80 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41670/0 (socket says 172.21.15.50:41670) 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36078/0 (socket says 172.21.15.50:36078) 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 -- 172.21.15.50:0/63865913 learned_addr learned my addr 172.21.15.50:0/63865913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:07.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0634da80 0x562f06351a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 -- 172.21.15.50:0/63865913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0634da80 msgr2=0x562f06351a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0634da80 0x562f06351a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 -- 172.21.15.50:0/63865913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 msgr2=0x562f06350490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69400700 1 -- 172.21.15.50:0/63865913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0633ec40 con 0x562f061c9e40 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.120+0000 7f0a69e00700 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.121+0000 7f0a69400700 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a58011040 tx=0x7f0a58005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78bfabe1fc5e196e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.121+0000 7f0a56a00700 1 -- 172.21.15.50:0/63865913 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5802e030 con 0x562f061c9e40 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.121+0000 7f0a56a00700 1 -- 172.21.15.50:0/63865913 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5800b790 con 0x562f061c9e40 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.121+0000 7f0a56a00700 1 -- 172.21.15.50:0/63865913 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58039040 con 0x562f061c9e40 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 -- 172.21.15.50:0/63865913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 msgr2=0x562f06347700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a58011040 tx=0x7f0a58005050 comp rx=0 tx=0).stop 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 -- 172.21.15.50:0/63865913 shutdown_connections 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f061c9e40 0x562f06347700 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0634da80 0x562f06351a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/63865913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06350490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 -- 172.21.15.50:0/63865913 >> 172.21.15.50:0/63865913 conn(0x562f0620ef70 msgr2=0x562f0634b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 -- 172.21.15.50:0/63865913 shutdown_connections 2024-05-10T21:14:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.122+0000 7f0a7f642b80 1 -- 172.21.15.50:0/63865913 wait complete. 2024-05-10T21:14:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.123+0000 7f0a7f642b80 1 -- start start 2024-05-10T21:14:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.123+0000 7f0a7f642b80 1 Processor -- start 2024-05-10T21:14:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f063554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06355f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1af190 con 0x562f06347c40 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1b05c0 con 0x562f0634da80 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c1af820 con 0x562f061c9e40 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a7f642b80 10 client.0 _ll_register_callbacks cb 0x562f06210c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36080/0 (socket says 172.21.15.50:36080) 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a69e00700 1 -- 172.21.15.50:0/2177633553 learned_addr learned my addr 172.21.15.50:0/2177633553 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a69400700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06355f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.124+0000 7f0a69400700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f063554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 -- 172.21.15.50:0/2177633553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 msgr2=0x562f063554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f063554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 -- 172.21.15.50:0/2177633553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 msgr2=0x562f06355f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06355f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 -- 172.21.15.50:0/2177633553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f06343d40 con 0x562f06347c40 2024-05-10T21:14:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a69e00700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011b50 tx=0x7f0a5c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2893b79451a479b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a4be00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.125+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0054e0 con 0x562f06347c40 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f0a5c003700 con 0x562f06347c40 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a5800b790 con 0x562f06347c40 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a5c011230 con 0x562f06347c40 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a7f642b80 10 client.16263 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f063cbe40 con 0x562f06347c40 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5c02d020 con 0x562f06347c40 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f0a5c01a910 con 0x562f06347c40 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a7f642b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.126+0000 7f0a7f642b80 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.127+0000 7f0a7f642b80 20 client.16263 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.127+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f063cc460 0x562f063ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.127+0000 7f0a7f642b80 4 client.16263 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.127+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f06277400 con 0x562f063cc460 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.127+0000 7f0a69400700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f063cc460 0x562f063ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a4be00700 10 client.16263 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a69400700 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f063cc460 0x562f063ce880 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89a4a03d0797e15d server_cookie=fd40eb826d4bb122 in_seq=0 out_seq=0 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a4be00700 1 -- 172.21.15.50:0/2177633553 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f06277400 con 0x562f063cc460 2024-05-10T21:14:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a4be00700 10 client.16263 handle_command_reply: tid=0 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 client.16263 shutdown 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 msgr2=0x562f06355a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011b50 tx=0x7f0a5c0093d0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 shutdown_connections 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f0634da80 0x562f06355f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x562f063cc460 0x562f063ce880 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f061c9e40 0x562f063554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 --2- 172.21.15.50:0/2177633553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f06347c40 0x562f06355a10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a4be00700 0 client.16263 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:14:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 >> 172.21.15.50:0/2177633553 conn(0x562f0620ef70 msgr2=0x562f0634b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.128+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 shutdown_connections 2024-05-10T21:14:07.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a7f642b80 1 -- 172.21.15.50:0/2177633553 wait complete. 2024-05-10T21:14:07.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a7f642b80 20 client.16263 trim_cache size 0 max 16384 2024-05-10T21:14:07.128 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 msgr2=0x7f0a500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 0x7f0a500b4d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0a640a9020 tx=0x7f0a64009040 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 msgr2=0x7f0a6c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.129+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400efc0 tx=0x7f0a6400c3a0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 shutdown_connections 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a6c104400 0x7f0a6c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a6c109680 0x7f0a6c076540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 --2- 172.21.15.50:0/1775673975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c108d30 0x7f0a6c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 >> 172.21.15.50:0/1775673975 conn(0x7f0a6c079d10 msgr2=0x7f0a6c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:07.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 shutdown_connections 2024-05-10T21:14:07.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:07.130+0000 7f0a6b200700 1 -- 172.21.15.50:0/1775673975 wait complete. 2024-05-10T21:14:07.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:06 smithi092 ceph-mon[34415]: pgmap v618: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 52 KiB/s wr, 40 op/s 2024-05-10T21:14:07.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:06 smithi142 ceph-mon[34846]: pgmap v618: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 52 KiB/s wr, 40 op/s 2024-05-10T21:14:08.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:14:08.540 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:14:08.996 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:08 smithi050 ceph-mon[29546]: pgmap v619: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-05-10T21:14:09.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:08 smithi092 ceph-mon[34415]: pgmap v619: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-05-10T21:14:09.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:08 smithi142 ceph-mon[34846]: pgmap v619: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 -- 172.21.15.50:0/2468742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10a7c0 msgr2=0x7f7f6c10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 --2- 172.21.15.50:0/2468742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c10abd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009990 tx=0x7f7f58009ca0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 -- 172.21.15.50:0/2468742320 shutdown_connections 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 --2- 172.21.15.50:0/2468742320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 --2- 172.21.15.50:0/2468742320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 --2- 172.21.15.50:0/2468742320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c10abd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.789+0000 7f7f6b200700 1 -- 172.21.15.50:0/2468742320 >> 172.21.15.50:0/2468742320 conn(0x7f7f6c076b00 msgr2=0x7f7f6c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:10.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.790+0000 7f7f6b200700 1 -- 172.21.15.50:0/2468742320 shutdown_connections 2024-05-10T21:14:10.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.790+0000 7f7f6b200700 1 -- 172.21.15.50:0/2468742320 wait complete. 2024-05-10T21:14:10.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.791+0000 7f7f6b200700 1 Processor -- start 2024-05-10T21:14:10.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.791+0000 7f7f6b200700 1 -- start start 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c1a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c11a900 con 0x7f7f6c10f6c0 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c11a780 con 0x7f7f6c10a7c0 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c11aa80 con 0x7f7f6c10b110 2024-05-10T21:14:10.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41692/0 (socket says 172.21.15.50:41692) 2024-05-10T21:14:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f69e00700 1 -- 172.21.15.50:0/3310645225 learned_addr learned my addr 172.21.15.50:0/3310645225 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:10.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f6a800700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c1a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.792+0000 7f7f69400700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 -- 172.21.15.50:0/3310645225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 msgr2=0x7f7f6c1a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 -- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 msgr2=0x7f7f6c1a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c1a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 -- 172.21.15.50:0/3310645225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1a6120 con 0x7f7f6c10b110 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f6a800700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c1a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f69400700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c1a17f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f580098e0 tx=0x7f7f580057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:10.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.793+0000 7f7f62a00700 1 -- 172.21.15.50:0/3310645225 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58027070 con 0x7f7f6c10b110 2024-05-10T21:14:10.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.794+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1a63b0 con 0x7f7f6c10b110 2024-05-10T21:14:10.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.794+0000 7f7f62a00700 1 -- 172.21.15.50:0/3310645225 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f58022470 con 0x7f7f6c10b110 2024-05-10T21:14:10.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.794+0000 7f7f62a00700 1 -- 172.21.15.50:0/3310645225 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58017650 con 0x7f7f6c10b110 2024-05-10T21:14:10.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.794+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1add30 con 0x7f7f6c10b110 2024-05-10T21:14:10.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.798+0000 7f7f62a00700 1 -- 172.21.15.50:0/3310645225 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f5802a070 con 0x7f7f6c10b110 2024-05-10T21:14:10.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.798+0000 7f7f7fbe5b80 1 Processor -- start 2024-05-10T21:14:10.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 -- start start 2024-05-10T21:14:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b445700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b446050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be3b367a20 con 0x55be3b44ba80 2024-05-10T21:14:10.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55be3b305760 con 0x55be3b445c40 2024-05-10T21:14:10.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.799+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be3b3639e0 con 0x55be3b2c7e40 2024-05-10T21:14:10.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f6a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b445700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f6a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41942/0 (socket says 172.21.15.50:41942) 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f6a800700 1 -- 172.21.15.50:0/1935675990 learned_addr learned my addr 172.21.15.50:0/1935675990 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f62a00700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 0x7f7f500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69400700 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b446050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 -- 172.21.15.50:0/1935675990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 msgr2=0x55be3b44b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 -- 172.21.15.50:0/1935675990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 msgr2=0x55be3b446050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b446050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f69e00700 1 -- 172.21.15.50:0/1935675990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3b2b48d0 con 0x55be3b44ba80 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f6a800700 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f62a00700 1 -- 172.21.15.50:0/3310645225 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7f580e30d0 con 0x7f7f6c10b110 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.800+0000 7f7f6a800700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 0x7f7f500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.801+0000 7f7f69e00700 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b445700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0114c0 tx=0x7f7f5c0117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c388dc598fc3cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.801+0000 7f7f57400700 1 -- 172.21.15.50:0/1935675990 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c004230 con 0x55be3b44ba80 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.801+0000 7f7f57400700 1 -- 172.21.15.50:0/1935675990 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c011e50 con 0x55be3b44ba80 2024-05-10T21:14:10.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.801+0000 7f7f57400700 1 -- 172.21.15.50:0/1935675990 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00e890 con 0x55be3b44ba80 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/1935675990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 msgr2=0x55be3b445700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f6a800700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 0x7f7f500b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f64003fd0 tx=0x7f7f64004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b445700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0114c0 tx=0x7f7f5c0117d0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/1935675990 shutdown_connections 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b446050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.802+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.803+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/1935675990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b445700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.803+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/1935675990 >> 172.21.15.50:0/1935675990 conn(0x55be3b30cf70 msgr2=0x55be3b440400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.803+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/1935675990 shutdown_connections 2024-05-10T21:14:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.803+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/1935675990 wait complete. 2024-05-10T21:14:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.803+0000 7f7f7fbe5b80 1 -- start start 2024-05-10T21:14:10.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 Processor -- start 2024-05-10T21:14:10.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 0x55be3b44f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b44ba80 0x55be3b44fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be3b452490 con 0x55be3b445c40 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55be3b2cdb20 con 0x55be3b44ba80 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be3b452610 con 0x55be3b2c7e40 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.804+0000 7f7f7fbe5b80 10 client.0 _ll_register_callbacks cb 0x55be3b30ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 0x55be3b44f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41944/0 (socket says 172.21.15.50:41944) 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 -- 172.21.15.50:0/3817112189 learned_addr learned my addr 172.21.15.50:0/3817112189 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f6a800700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b44ba80 0x55be3b44fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 -- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b44ba80 msgr2=0x55be3b44fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b44ba80 0x55be3b44fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 -- 172.21.15.50:0/3817112189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 msgr2=0x55be3b44f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 0x55be3b44f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:10.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 -- 172.21.15.50:0/3817112189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3b32d420 con 0x55be3b2c7e40 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69400700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 0x55be3b44f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005e90 tx=0x7f7f5c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614ec4836f5582ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.805+0000 7f7f4be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c005660 con 0x55be3b2c7e40 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c011e50 con 0x55be3b2c7e40 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7f5c003680 con 0x55be3b2c7e40 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c031070 con 0x55be3b2c7e40 2024-05-10T21:14:10.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f7fbe5b80 10 client.26365 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.806+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55be3b45d000 con 0x55be3b2c7e40 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c011e50 con 0x55be3b2c7e40 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f7f5c00e580 con 0x55be3b2c7e40 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 20 client.26365 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:10.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b45f590 0x55be3b4619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 4 client.26365 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:14:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3b4121c0 con 0x55be3b45f590 2024-05-10T21:14:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.807+0000 7f7f69400700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b45f590 0x55be3b4619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.808+0000 7f7f69400700 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b45f590 0x55be3b4619b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d41c9995f2f9b99 server_cookie=ab0e77a15968f5d6 in_seq=0 out_seq=0 2024-05-10T21:14:10.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:10.808+0000 7f7f4be00700 10 client.26365 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:14:11.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:10 smithi050 ceph-mon[29546]: pgmap v620: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-10T21:14:11.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:10 smithi092 ceph-mon[34415]: pgmap v620: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-10T21:14:11.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:10 smithi142 ceph-mon[34846]: pgmap v620: 97 pgs: 97 active+clean; 285 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-10T21:14:11.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 54885 0 35237 0 16176 0 2024-05-10T21:14:13.179 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:12 smithi142 ceph-mon[34846]: pgmap v621: 97 pgs: 97 active+clean; 286 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-05-10T21:14:13.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:12 smithi092 ceph-mon[34415]: pgmap v621: 97 pgs: 97 active+clean; 286 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-05-10T21:14:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:12 smithi050 ceph-mon[29546]: pgmap v621: 97 pgs: 97 active+clean; 286 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-05-10T21:14:15.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:14 smithi092 ceph-mon[34415]: pgmap v622: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 197 KiB/s wr, 14 op/s 2024-05-10T21:14:15.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:14 smithi142 ceph-mon[34846]: pgmap v622: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 197 KiB/s wr, 14 op/s 2024-05-10T21:14:15.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:14 smithi050 ceph-mon[29546]: pgmap v622: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 197 KiB/s wr, 14 op/s 2024-05-10T21:14:17.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:16 smithi092 ceph-mon[34415]: pgmap v623: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 206 KiB/s wr, 15 op/s 2024-05-10T21:14:17.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:16 smithi142 ceph-mon[34846]: pgmap v623: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 206 KiB/s wr, 15 op/s 2024-05-10T21:14:17.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:16 smithi050 ceph-mon[29546]: pgmap v623: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 206 KiB/s wr, 15 op/s 2024-05-10T21:14:19.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:18 smithi092 ceph-mon[34415]: pgmap v624: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 205 KiB/s wr, 15 op/s 2024-05-10T21:14:19.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:14:19.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:18 smithi142 ceph-mon[34846]: pgmap v624: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 205 KiB/s wr, 15 op/s 2024-05-10T21:14:19.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:18 smithi050 ceph-mon[29546]: pgmap v624: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 205 KiB/s wr, 15 op/s 2024-05-10T21:14:19.846 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:14:21.051 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:20 smithi050 ceph-mon[29546]: pgmap v625: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 205 KiB/s wr, 13 op/s 2024-05-10T21:14:21.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:20 smithi092 ceph-mon[34415]: pgmap v625: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 205 KiB/s wr, 13 op/s 2024-05-10T21:14:21.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:20 smithi142 ceph-mon[34846]: pgmap v625: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 205 KiB/s wr, 13 op/s 2024-05-10T21:14:21.400 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:14:22.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.254+0000 7f1d74a00700 1 -- 172.21.15.50:0/2168068208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70076450 msgr2=0x7f1d701106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:22.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.254+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2168068208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70076450 0x7f1d701106c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009990 tx=0x7f1d5c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 -- 172.21.15.50:0/2168068208 shutdown_connections 2024-05-10T21:14:22.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2168068208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70110c00 0x7f1d70113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2168068208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70075b00 0x7f1d70075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2168068208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70076450 0x7f1d701106c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 -- 172.21.15.50:0/2168068208 >> 172.21.15.50:0/2168068208 conn(0x7f1d70100040 msgr2=0x7f1d70102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 -- 172.21.15.50:0/2168068208 shutdown_connections 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.255+0000 7f1d74a00700 1 -- 172.21.15.50:0/2168068208 wait complete. 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.256+0000 7f1d74a00700 1 Processor -- start 2024-05-10T21:14:22.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.256+0000 7f1d74a00700 1 -- start start 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 0x7f1d7019d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70110c00 0x7f1d701a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70116710 con 0x7f1d70076450 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70116590 con 0x7f1d70110c00 2024-05-10T21:14:22.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d74a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70116890 con 0x7f1d70075b00 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d6d400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d6d400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45170/0 (socket says 172.21.15.50:45170) 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d6de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 0x7f1d7019d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d6d400700 1 -- 172.21.15.50:0/2360107223 learned_addr learned my addr 172.21.15.50:0/2360107223 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.257+0000 7f1d6e800700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70110c00 0x7f1d701a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 -- 172.21.15.50:0/2360107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70110c00 msgr2=0x7f1d701a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70110c00 0x7f1d701a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 -- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 msgr2=0x7f1d7019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 -- 172.21.15.50:0/2360107223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d701a1f90 con 0x7f1d70075b00 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6d400700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d6de00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 0x7f1d7019d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e9f0 tx=0x7f1d6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:22.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000c9c0 con 0x7f1d70075b00 2024-05-10T21:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6000cb60 con 0x7f1d70075b00 2024-05-10T21:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701a2280 con 0x7f1d70075b00 2024-05-10T21:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.258+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60018680 con 0x7f1d70075b00 2024-05-10T21:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.259+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d701a9ba0 con 0x7f1d70075b00 2024-05-10T21:14:22.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.260+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d700512c0 con 0x7f1d70075b00 2024-05-10T21:14:22.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.262+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d60010430 con 0x7f1d70075b00 2024-05-10T21:14:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.264+0000 7f1d66a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 0x7f1d540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.264+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f1d600e2710 con 0x7f1d70075b00 2024-05-10T21:14:22.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.264+0000 7f1d6d400700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 0x7f1d540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:22.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.265+0000 7f1d6d400700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 0x7f1d540b4d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d70103470 tx=0x7f1d5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:22.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.266+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d600aa400 con 0x7f1d70075b00 2024-05-10T21:14:22.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.552+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7002e5f0 con 0x7f1d70075b00 2024-05-10T21:14:22.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.554+0000 7f1d66a00700 1 -- 172.21.15.50:0/2360107223 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11856 (secure 0 0 0) 0x7f1d600a9d00 con 0x7f1d70075b00 2024-05-10T21:14:22.555 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:14:22.555 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:13:39.382028+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24502,24520,24539]},"id":1}]} 2024-05-10T21:14:22.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.557+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 msgr2=0x7f1d540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.557+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 0x7f1d540b4d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d70103470 tx=0x7f1d5c005fd0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.557+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 msgr2=0x7f1d7019d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.557+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 0x7f1d7019d0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e9f0 tx=0x7f1d6000ed00 comp rx=0 tx=0).stop 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 shutdown_connections 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f1d540b2880 0x7f1d540b4d90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d70110c00 0x7f1d701a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70075b00 0x7f1d7019d0c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 --2- 172.21.15.50:0/2360107223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d70076450 0x7f1d7019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 >> 172.21.15.50:0/2360107223 conn(0x7f1d70100040 msgr2=0x7f1d701006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 shutdown_connections 2024-05-10T21:14:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:22.558+0000 7f1d74a00700 1 -- 172.21.15.50:0/2360107223 wait complete. 2024-05-10T21:14:22.558 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 32 2024-05-10T21:14:23.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:22 smithi050 ceph-mon[29546]: pgmap v626: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-05-10T21:14:23.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:22 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2360107223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:14:23.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:22 smithi092 ceph-mon[34415]: pgmap v626: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-05-10T21:14:23.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:22 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2360107223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:14:23.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:22 smithi142 ceph-mon[34846]: pgmap v626: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-05-10T21:14:23.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:22 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2360107223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:14:23.412 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --set-id f0125e47 --release --await' 2024-05-10T21:14:23.413 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db --set-id f0125e47 --release --await 2024-05-10T21:14:23.931 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:14:25.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:24 smithi092 ceph-mon[34415]: pgmap v627: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 102 KiB/s wr, 10 op/s 2024-05-10T21:14:25.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:24 smithi142 ceph-mon[34846]: pgmap v627: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 102 KiB/s wr, 10 op/s 2024-05-10T21:14:25.383 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:24 smithi050 ceph-mon[29546]: pgmap v627: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 102 KiB/s wr, 10 op/s 2024-05-10T21:14:26.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.340+0000 7fe3b8e00700 1 -- 172.21.15.50:0/895092582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 msgr2=0x7fe3b4117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:26.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.340+0000 7fe3b8e00700 1 --2- 172.21.15.50:0/895092582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 0x7fe3b4117620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00b0a0 tx=0x7fe3ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 -- 172.21.15.50:0/895092582 shutdown_connections 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 --2- 172.21.15.50:0/895092582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 0x7fe3b4117620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 --2- 172.21.15.50:0/895092582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b410fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 --2- 172.21.15.50:0/895092582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 0x7fe3b410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 -- 172.21.15.50:0/895092582 >> 172.21.15.50:0/895092582 conn(0x7fe3b4076b00 msgr2=0x7fe3b4076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 -- 172.21.15.50:0/895092582 shutdown_connections 2024-05-10T21:14:26.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.341+0000 7fe3b8e00700 1 -- 172.21.15.50:0/895092582 wait complete. 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.342+0000 7fe3b8e00700 1 Processor -- start 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.342+0000 7fe3b8e00700 1 -- start start 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 0x7fe3b41a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 0x7fe3b41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b41a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b411a8f0 con 0x7fe3b410f6c0 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b411a770 con 0x7fe3b410a7c0 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b8e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b411aa70 con 0x7fe3b410b110 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b41a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b41a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60314/0 (socket says 172.21.15.50:60314) 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b3e00700 1 -- 172.21.15.50:0/2754925740 learned_addr learned my addr 172.21.15.50:0/2754925740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 0x7fe3b41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.343+0000 7fe3b3400700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 0x7fe3b41a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 -- 172.21.15.50:0/2754925740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 msgr2=0x7fe3b41a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 0x7fe3b41a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 -- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 msgr2=0x7fe3b41a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b41a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 -- 172.21.15.50:0/2754925740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b41a60b0 con 0x7fe3b410b110 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b3400700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b410a7c0 0x7fe3b41a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3b410f6c0 0x7fe3b41a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:14:26.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b410b110 0x7fe3b41a1780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00e9f0 tx=0x7fe39c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.344+0000 7fe3abe00700 1 -- 172.21.15.50:0/2754925740 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00c9c0 con 0x7fe3b410b110 2024-05-10T21:14:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.345+0000 7fe3abe00700 1 -- 172.21.15.50:0/2754925740 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c00cb60 con 0x7fe3b410b110 2024-05-10T21:14:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.345+0000 7fe3abe00700 1 -- 172.21.15.50:0/2754925740 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c018680 con 0x7fe3b410b110 2024-05-10T21:14:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.345+0000 7fe3b8e00700 1 -- 172.21.15.50:0/2754925740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b41a63a0 con 0x7fe3b410b110 2024-05-10T21:14:26.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.345+0000 7fe3b8e00700 1 -- 172.21.15.50:0/2754925740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b41adcc0 con 0x7fe3b410b110 2024-05-10T21:14:26.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.351+0000 7fe3abe00700 1 -- 172.21.15.50:0/2754925740 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe39c010430 con 0x7fe3b410b110 2024-05-10T21:14:26.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.351+0000 7fe3c9010b80 1 Processor -- start 2024-05-10T21:14:26.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.351+0000 7fe3c9010b80 1 -- start start 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5578fc1c6b20 con 0x5578fc2a4900 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578fc164860 con 0x5578fc2a44f0 2024-05-10T21:14:26.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5578fc1c2ae0 con 0x5578fc126f40 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57634/0 (socket says 172.21.15.50:57634) 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3b3e00700 1 -- 172.21.15.50:0/3227575946 learned_addr learned my addr 172.21.15.50:0/3227575946 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3b3400700 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.352+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 -- 172.21.15.50:0/3227575946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a44f0 msgr2=0x5578fc2aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 -- 172.21.15.50:0/3227575946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 msgr2=0x5578fc2b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 -- 172.21.15.50:0/3227575946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578fc1139d0 con 0x5578fc126f40 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3abe00700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3980b2900 0x7fe3980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d010 tx=0x7fe3ac009db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb7d0a070e4d10e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3abe00700 1 -- 172.21.15.50:0/2754925740 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe39c0e2320 con 0x7fe3b410b110 2024-05-10T21:14:26.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.353+0000 7fe3b3400700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3980b2900 0x7fe3980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.354+0000 7fe3a8c00700 1 -- 172.21.15.50:0/3227575946 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac021070 con 0x5578fc126f40 2024-05-10T21:14:26.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.354+0000 7fe3a8c00700 1 -- 172.21.15.50:0/3227575946 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac004c50 con 0x5578fc126f40 2024-05-10T21:14:26.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.354+0000 7fe3a8c00700 1 -- 172.21.15.50:0/3227575946 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0052a0 con 0x5578fc126f40 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 -- 172.21.15.50:0/3227575946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 msgr2=0x5578fc2a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d010 tx=0x7fe3ac009db0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3b3400700 1 --2- 172.21.15.50:0/2754925740 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe3980b2900 0x7fe3980b4e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5578fc2a5510 tx=0x7fe3a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 -- 172.21.15.50:0/3227575946 shutdown_connections 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc126f40 0x5578fc2a3fb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/3227575946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 -- 172.21.15.50:0/3227575946 >> 172.21.15.50:0/3227575946 conn(0x5578fc16c070 msgr2=0x5578fc29fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 -- 172.21.15.50:0/3227575946 shutdown_connections 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.355+0000 7fe3c9010b80 1 -- 172.21.15.50:0/3227575946 wait complete. 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 -- start start 2024-05-10T21:14:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 Processor -- start 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5578fc2b1a10 con 0x5578fc2a4900 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578fc12cc20 con 0x5578fc126f40 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.356+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5578fc2b1b90 con 0x5578fc2a44f0 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3c9010b80 10 client.0 _ll_register_callbacks cb 0x5578fc16dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43168/0 (socket says 172.21.15.50:43168) 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 -- 172.21.15.50:0/1481950453 learned_addr learned my addr 172.21.15.50:0/1481950453 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 -- 172.21.15.50:0/1481950453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 msgr2=0x5578fc2aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 -- 172.21.15.50:0/1481950453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 msgr2=0x5578fc2af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.357+0000 7fe3b3400700 1 -- 172.21.15.50:0/1481950453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578fc18c520 con 0x5578fc126f40 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3a401da80 tx=0x7fe3a40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba201a3e3add097 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3a1600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a402d060 con 0x5578fc126f40 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe3a4018030 con 0x5578fc126f40 2024-05-10T21:14:26.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ac004c50 con 0x5578fc126f40 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3a401dd90 con 0x5578fc126f40 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3c9010b80 10 client.26432 fetch_fsmap learned FSMap version 32 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.358+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5578fc2bc720 con 0x5578fc126f40 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac004c50 con 0x5578fc126f40 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe3a40338d0 con 0x5578fc126f40 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 10 client.26432 fetch_fsmap finished waiting for FSMap version 32 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 10 client.26432 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 20 client.26432 populate_metadata read hostname 'smithi050' 2024-05-10T21:14:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc2bed60 0x5578fc2c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:14:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 4 client.26432 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:14:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578fc2712c0 con 0x5578fc2bed60 2024-05-10T21:14:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.359+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc2bed60 0x5578fc2c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:14:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.360+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc2bed60 0x5578fc2c11c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ef6631867f83e4 server_cookie=937d707e52d5a2e6 in_seq=0 out_seq=0 2024-05-10T21:14:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:14:26.360+0000 7fe3a1600700 10 client.26432 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:14:27.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:26 smithi092 ceph-mon[34415]: pgmap v628: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-10T21:14:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:26 smithi142 ceph-mon[34846]: pgmap v628: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-10T21:14:27.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:26 smithi050 ceph-mon[29546]: pgmap v628: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-10T21:14:29.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:28 smithi092 ceph-mon[34415]: pgmap v629: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-10T21:14:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:28 smithi142 ceph-mon[34846]: pgmap v629: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-10T21:14:29.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:28 smithi050 ceph-mon[29546]: pgmap v629: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-10T21:14:31.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:30 smithi092 ceph-mon[34415]: pgmap v630: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-05-10T21:14:31.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:30 smithi142 ceph-mon[34846]: pgmap v630: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-05-10T21:14:31.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:14:31.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:30 smithi050 ceph-mon[29546]: pgmap v630: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-05-10T21:14:33.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:32 smithi092 ceph-mon[34415]: pgmap v631: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-10T21:14:33.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:32 smithi142 ceph-mon[34846]: pgmap v631: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-10T21:14:33.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:32 smithi050 ceph-mon[29546]: pgmap v631: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-10T21:14:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:34 smithi092 ceph-mon[34415]: pgmap v632: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T21:14:35.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:34 smithi142 ceph-mon[34846]: pgmap v632: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T21:14:35.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:34 smithi050 ceph-mon[29546]: pgmap v632: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T21:14:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:36 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:14:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:36 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:14:36.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:36 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:14:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:37 smithi092 ceph-mon[34415]: pgmap v633: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:14:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:37 smithi142 ceph-mon[34846]: pgmap v633: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:14:37.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:37 smithi050 ceph-mon[29546]: pgmap v633: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T21:14:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:39 smithi092 ceph-mon[34415]: pgmap v634: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T21:14:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:39 smithi142 ceph-mon[34846]: pgmap v634: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T21:14:39.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:39 smithi050 ceph-mon[29546]: pgmap v634: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-10T21:14:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:41 smithi092 ceph-mon[34415]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:14:41.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:41 smithi142 ceph-mon[34846]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:14:41.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:14:41.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:41 smithi050 ceph-mon[29546]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:14:42.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:42 smithi092 ceph-mon[34415]: pgmap v636: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-10T21:14:42.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:42 smithi142 ceph-mon[34846]: pgmap v636: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-10T21:14:42.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:42 smithi050 ceph-mon[29546]: pgmap v636: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-10T21:14:45.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:44 smithi050 ceph-mon[29546]: pgmap v637: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-10T21:14:45.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:44 smithi092 ceph-mon[34415]: pgmap v637: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-10T21:14:45.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:44 smithi142 ceph-mon[34846]: pgmap v637: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-10T21:14:47.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:46 smithi050 ceph-mon[29546]: pgmap v638: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-10T21:14:47.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:46 smithi092 ceph-mon[34415]: pgmap v638: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-10T21:14:47.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:46 smithi142 ceph-mon[34846]: pgmap v638: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-05-10T21:14:48.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:48 smithi092 ceph-mon[34415]: pgmap v639: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-10T21:14:48.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:48 smithi142 ceph-mon[34846]: pgmap v639: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-10T21:14:48.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:48 smithi050 ceph-mon[29546]: pgmap v639: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-10T21:14:51.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:50 smithi050 ceph-mon[29546]: pgmap v640: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-05-10T21:14:51.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:50 smithi092 ceph-mon[34415]: pgmap v640: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-05-10T21:14:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:50 smithi142 ceph-mon[34846]: pgmap v640: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-05-10T21:14:51.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:14:53.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:52 smithi050 ceph-mon[29546]: pgmap v641: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-10T21:14:53.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:52 smithi092 ceph-mon[34415]: pgmap v641: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-10T21:14:53.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:52 smithi142 ceph-mon[34846]: pgmap v641: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-10T21:14:55.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:54 smithi050 ceph-mon[29546]: pgmap v642: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-10T21:14:55.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:54 smithi092 ceph-mon[34415]: pgmap v642: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-10T21:14:55.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:54 smithi142 ceph-mon[34846]: pgmap v642: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-10T21:14:56.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:55 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:14:56.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:55 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:14:56.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:55 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:14:57.269 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:56 smithi092 ceph-mon[34415]: pgmap v643: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-10T21:14:57.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:56 smithi142 ceph-mon[34846]: pgmap v643: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-10T21:14:57.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:56 smithi050 ceph-mon[29546]: pgmap v643: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-10T21:14:58.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:57 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:14:58.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:57 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:14:58.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:57 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:14:58.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:14:58.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:14:58.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:14:58.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:14:58.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:14:58.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:14:59.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:14:58 smithi092 ceph-mon[34415]: pgmap v644: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-10T21:14:59.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:14:58 smithi142 ceph-mon[34846]: pgmap v644: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-10T21:14:59.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:14:58 smithi050 ceph-mon[29546]: pgmap v644: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-10T21:15:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:00 smithi092 ceph-mon[34415]: pgmap v645: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:15:01.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:00 smithi142 ceph-mon[34846]: pgmap v645: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:15:01.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:15:01.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:00 smithi050 ceph-mon[29546]: pgmap v645: 97 pgs: 97 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-10T21:15:03.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:02 smithi092 ceph-mon[34415]: pgmap v646: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-10T21:15:03.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:02 smithi142 ceph-mon[34846]: pgmap v646: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-10T21:15:03.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:02 smithi050 ceph-mon[29546]: pgmap v646: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-10T21:15:05.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:04 smithi092 ceph-mon[34415]: pgmap v647: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-05-10T21:15:05.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:04 smithi142 ceph-mon[34846]: pgmap v647: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-05-10T21:15:05.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:04 smithi050 ceph-mon[29546]: pgmap v647: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-05-10T21:15:07.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:06 smithi092 ceph-mon[34415]: pgmap v648: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:07.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:06 smithi142 ceph-mon[34846]: pgmap v648: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:07.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:06 smithi050 ceph-mon[29546]: pgmap v648: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:09.218 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:08 smithi050 ceph-mon[29546]: pgmap v649: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-05-10T21:15:09.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:08 smithi092 ceph-mon[34415]: pgmap v649: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-05-10T21:15:09.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:08 smithi142 ceph-mon[34846]: pgmap v649: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-05-10T21:15:11.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:10 smithi092 ceph-mon[34415]: pgmap v650: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-05-10T21:15:11.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:10 smithi142 ceph-mon[34846]: pgmap v650: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-05-10T21:15:11.399 INFO:tasks.workunit.client.0.smithi050.stdout: 68 0 0 0 0 0 0 2024-05-10T21:15:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:10 smithi050 ceph-mon[29546]: pgmap v650: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-05-10T21:15:13.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:12 smithi092 ceph-mon[34415]: pgmap v651: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:13.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:12 smithi142 ceph-mon[34846]: pgmap v651: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:12 smithi050 ceph-mon[29546]: pgmap v651: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-05-10T21:15:15.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:14 smithi092 ceph-mon[34415]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 KiB/s wr, 6 op/s 2024-05-10T21:15:15.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:14 smithi142 ceph-mon[34846]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 KiB/s wr, 6 op/s 2024-05-10T21:15:15.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:14 smithi050 ceph-mon[29546]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.4 KiB/s wr, 6 op/s 2024-05-10T21:15:17.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:16 smithi092 ceph-mon[34415]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-10T21:15:17.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:16 smithi142 ceph-mon[34846]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-10T21:15:17.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:16 smithi050 ceph-mon[29546]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-10T21:15:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.988+0000 7f2379600640 1 -- 172.21.15.50:0/3200105236 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afdd708fd0 con 0x55afdd38f1c0 2024-05-10T21:15:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.988+0000 7f2379600640 10 client.16221 handle_command_reply: tid=0 2024-05-10T21:15:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.988+0000 7f23a1a25740 1 client.16221 shutdown 2024-05-10T21:15:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.989+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 msgr2=0x55afdd81b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.989+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f239412d040 tx=0x7f2394064250 comp rx=0 tx=0).stop 2024-05-10T21:15:18.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7f7f4be00700 1 -- 172.21.15.50:0/3817112189 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be3b4121c0 con 0x55be3b45f590 2024-05-10T21:15:18.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7f7f4be00700 10 client.26365 handle_command_reply: tid=0 2024-05-10T21:15:18.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7f7f7fbe5b80 1 client.26365 shutdown 2024-05-10T21:15:18.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 msgr2=0x55be3b44f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:18.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005e90 tx=0x7f7f5c005270 comp rx=0 tx=0).stop 2024-05-10T21:15:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 shutdown_connections 2024-05-10T21:15:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55afdd38f1c0 0x55afdd827b60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afdd3fb1b0 0x55afdd81b240 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55afdd3da4d0 0x55afdd7adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 --2- 172.21.15.50:0/3200105236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55afdd3d22f0 0x55afdd7ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 >> 172.21.15.50:0/3200105236 conn(0x55afdd3ef300 msgr2=0x55afdd79c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:18.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7fe3a1600700 1 -- 172.21.15.50:0/1481950453 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578fc2712c0 con 0x5578fc2bed60 2024-05-10T21:15:18.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7fe3a1600700 10 client.26432 handle_command_reply: tid=0 2024-05-10T21:15:18.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.989+0000 7fe3c9010b80 1 client.26432 shutdown 2024-05-10T21:15:18.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 msgr2=0x5578fc2ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:18.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3a401da80 tx=0x7fe3a40047c0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f2379600640 0 client.16221 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.991+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 shutdown_connections 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 shutdown_connections 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc2bed60 0x5578fc2c11c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc126f40 0x5578fc2ae670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc2a4900 0x5578fc2af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1481950453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 >> 172.21.15.50:0/1481950453 conn(0x5578fc16c070 msgr2=0x5578fc2a5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.991+0000 7fe3a1600700 0 client.26432 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 shutdown_connections 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b45f590 0x55be3b4619b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b44ba80 0x55be3b44fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b44f130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.991+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 shutdown_connections 2024-05-10T21:15:18.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3817112189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b445c40 0x55be3b44f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 >> 172.21.15.50:0/3817112189 conn(0x55be3b30cf70 msgr2=0x55be3b4446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:18.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f4be00700 0 client.26365 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:18.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.991+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1481950453 wait complete. 2024-05-10T21:15:18.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 shutdown_connections 2024-05-10T21:15:18.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.991+0000 7fe3c9010b80 20 client.26432 trim_cache size 0 max 16384 2024-05-10T21:15:18.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.990+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3817112189 wait complete. 2024-05-10T21:15:18.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:18.991+0000 7f7f7fbe5b80 20 client.26365 trim_cache size 0 max 16384 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.993+0000 7f23a1a25740 1 -- 172.21.15.50:0/3200105236 wait complete. 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.993+0000 7f23a1a25740 20 client.16221 trim_cache size 0 max 16384 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:15:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:15:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.994+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 msgr2=0x7f237c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.994+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 0x7f237c0b4d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f238802aa00 tx=0x7f23880314a0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.994+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 msgr2=0x7f238c136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.994+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2394065d90 tx=0x7f239407b6a0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 shutdown_connections 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f237c0b2870 0x7f237c0b4d80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f238c09eb10 0x7f238c136810 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238c099e10 0x7f238c12f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 --2- 172.21.15.50:0/3366101431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f238c098bd0 0x7f238c12f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 >> 172.21.15.50:0/3366101431 conn(0x7f238c093de0 msgr2=0x7f238c09aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.995+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 shutdown_connections 2024-05-10T21:15:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:18.996+0000 7f2399400640 1 -- 172.21.15.50:0/3366101431 wait complete. 2024-05-10T21:15:19.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.024+0000 7f7f7fbe5b80 1 Processor -- start 2024-05-10T21:15:19.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.024+0000 7f7f7fbe5b80 1 -- start start 2024-05-10T21:15:19.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b44ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b44f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b7810 con 0x55be3b44ba80 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b5010 con 0x55be3b445c40 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b2ee0 con 0x55be3b2c7e40 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56694/0 (socket says 172.21.15.50:56694) 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f69e00700 1 -- 172.21.15.50:0/2913426008 learned_addr learned my addr 172.21.15.50:0/2913426008 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f69400700 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b44ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.025+0000 7f7f69e00700 1 -- 172.21.15.50:0/2913426008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 msgr2=0x55be3b44ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f69e00700 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b44ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f69e00700 1 -- 172.21.15.50:0/2913426008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 msgr2=0x55be3b44f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f69e00700 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b44f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f69e00700 1 -- 172.21.15.50:0/2913426008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3b43cc40 con 0x55be3b2c7e40 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f69e00700 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c011810 tx=0x7f7f5c03a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=378a3881c8df97d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f56a00700 1 -- 172.21.15.50:0/2913426008 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c012e60 con 0x55be3b2c7e40 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.026+0000 7f7f56a00700 1 -- 172.21.15.50:0/2913426008 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c042070 con 0x55be3b2c7e40 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.028+0000 7f7f56a00700 1 -- 172.21.15.50:0/2913426008 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c03a070 con 0x55be3b2c7e40 2024-05-10T21:15:19.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.031+0000 7fe3c9010b80 1 Processor -- start 2024-05-10T21:15:19.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.031+0000 7fe3c9010b80 1 -- start start 2024-05-10T21:15:19.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.029+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/2913426008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 msgr2=0x55be3b445700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.029+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c011810 tx=0x7f7f5c03a000 comp rx=0 tx=0).stop 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.030+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/2913426008 shutdown_connections 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b76e0 con 0x5578fc126f40 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b4ee0 con 0x5578fc2a4900 2024-05-10T21:15:19.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b2d90 con 0x5578fc2a44f0 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.030+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b44f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.030+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b2c7e40 0x55be3b445700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.030+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/2913426008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b445c40 0x55be3b44ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.030+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/2913426008 >> 172.21.15.50:0/2913426008 conn(0x55be3b30cf70 msgr2=0x55be3b43a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.031+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/2913426008 shutdown_connections 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.031+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/2913426008 wait complete. 2024-05-10T21:15:19.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.032+0000 7f7f7fbe5b80 1 -- start start 2024-05-10T21:15:19.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7f7f7fbe5b80 1 Processor -- start 2024-05-10T21:15:19.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34418/0 (socket says 172.21.15.50:34418) 2024-05-10T21:15:19.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7fe3b3400700 1 -- 172.21.15.50:0/1536938041 learned_addr learned my addr 172.21.15.50:0/1536938041 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:19.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7fe3b3400700 1 -- 172.21.15.50:0/1536938041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 msgr2=0x5578fc2aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.033+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7fe3b3400700 1 -- 172.21.15.50:0/1536938041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 msgr2=0x5578fc2baaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2baaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7fe3b3400700 1 -- 172.21.15.50:0/1536938041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578fc2acbe0 con 0x5578fc126f40 2024-05-10T21:15:19.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.036+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:19.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 0x55be3b457ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b452c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b3530 con 0x55be3b44ba80 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b4940 con 0x55be3b2c7e40 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b3ba0 con 0x55be3b445c40 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f7fbe5b80 10 client.0 _ll_register_callbacks cb 0x55be3b30ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:19.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:46382/0 (socket says 172.21.15.50:46382) 2024-05-10T21:15:19.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.034+0000 7f7f69e00700 1 -- 172.21.15.50:0/3004255722 learned_addr learned my addr 172.21.15.50:0/3004255722 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:19.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69e00700 1 -- 172.21.15.50:0/3004255722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 msgr2=0x55be3b457ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.037+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:19.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69400700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 0x55be3b457ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 0x55be3b457ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69e00700 1 -- 172.21.15.50:0/3004255722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 msgr2=0x55be3b452c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b452c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69e00700 1 -- 172.21.15.50:0/3004255722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3b453350 con 0x55be3b2c7e40 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.035+0000 7f7f69400700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 0x55be3b457ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.036+0000 7f7f69e00700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00eb10 tx=0x7f7f5c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5870a4b434d95afe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.036+0000 7f7f4be00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.036+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c003c90 con 0x55be3b2c7e40 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.036+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f7f5c005250 con 0x55be3b2c7e40 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.037+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c042070 con 0x55be3b2c7e40 2024-05-10T21:15:19.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.037+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4019040 tx=0x7fe3a403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6495cc55bb0fbd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:19.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.038+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c04c050 con 0x55be3b2c7e40 2024-05-10T21:15:19.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.038+0000 7f7f7fbe5b80 10 client.26438 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:19.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.038+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55be3b4c9470 con 0x55be3b2c7e40 2024-05-10T21:15:19.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.038+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c042070 con 0x55be3b2c7e40 2024-05-10T21:15:19.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.038+0000 7fe3a3e00700 1 -- 172.21.15.50:0/1536938041 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4040070 con 0x5578fc126f40 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7f5c036570 con 0x55be3b2c7e40 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 10 client.26438 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 20 client.26438 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b4c9a00 0x55be3b4cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 4 client.26438 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:15:19.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be3b375400 con 0x55be3b4c9a00 2024-05-10T21:15:19.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7fe3a3e00700 1 -- 172.21.15.50:0/1536938041 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a40044f0 con 0x5578fc126f40 2024-05-10T21:15:19.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.039+0000 7fe3a3e00700 1 -- 172.21.15.50:0/1536938041 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4038070 con 0x5578fc126f40 2024-05-10T21:15:19.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.040+0000 7f7f69400700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b4c9a00 0x55be3b4cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7f7f69400700 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b4c9a00 0x55be3b4cbe20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6297320fde8864b server_cookie=75753754ef343e4c in_seq=0 out_seq=0 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1536938041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 msgr2=0x5578fc2ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4019040 tx=0x7fe3a403d000 comp rx=0 tx=0).stop 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7f7f4be00700 10 client.26438 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7f7f4be00700 1 -- 172.21.15.50:0/3004255722 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be3b375400 con 0x55be3b4c9a00 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7f7f4be00700 10 client.26438 handle_command_reply: tid=0 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1536938041 shutdown_connections 2024-05-10T21:15:19.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2ad5d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1536938041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.041+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1536938041 >> 172.21.15.50:0/1536938041 conn(0x5578fc16c070 msgr2=0x5578fc2afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1536938041 shutdown_connections 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1536938041 wait complete. 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.042+0000 7f7f7fbe5b80 1 client.26438 shutdown 2024-05-10T21:15:19.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.043+0000 7fe3c9010b80 1 -- start start 2024-05-10T21:15:19.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.044+0000 7fe3c9010b80 1 Processor -- start 2024-05-10T21:15:19.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.044+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 msgr2=0x55be3b457760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.044+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00eb10 tx=0x7f7f5c00ef10 comp rx=0 tx=0).stop 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b33e0 con 0x5578fc126f40 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b4810 con 0x5578fc2a4900 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41b3a70 con 0x5578fc2a44f0 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7fe3c9010b80 10 client.0 _ll_register_callbacks cb 0x5578fc16dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34420/0 (socket says 172.21.15.50:34420) 2024-05-10T21:15:19.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b3400700 1 -- 172.21.15.50:0/1243263242 learned_addr learned my addr 172.21.15.50:0/1243263242 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:19.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 shutdown_connections 2024-05-10T21:15:19.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55be3b4c9a00 0x55be3b4cbe20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55be3b2c7e40 0x55be3b457760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be3b44ba80 0x55be3b452c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 --2- 172.21.15.50:0/3004255722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3b445c40 0x55be3b457ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 >> 172.21.15.50:0/3004255722 conn(0x55be3b30cf70 msgr2=0x55be3b44f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.045+0000 7f7f4be00700 0 client.26438 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 shutdown_connections 2024-05-10T21:15:19.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7f7f7fbe5b80 1 -- 172.21.15.50:0/3004255722 wait complete. 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.046+0000 7fe3b3e00700 1 -- 172.21.15.50:0/1243263242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 msgr2=0x5578fc2b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3e00700 1 -- 172.21.15.50:0/1243263242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 msgr2=0x5578fc2b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3e00700 1 -- 172.21.15.50:0/1243263242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578fc2b37b0 con 0x5578fc2a4900 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b2a00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3b3e00700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2b3100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009900 tx=0x7fe3ac0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c734f8cc30a3abcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.047+0000 7fe3a1600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.048+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac032060 con 0x5578fc2a4900 2024-05-10T21:15:19.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.048+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe3ac0052d0 con 0x5578fc2a4900 2024-05-10T21:15:19.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.048+0000 7f7f7fbe5b80 20 client.26438 trim_cache size 0 max 16384 2024-05-10T21:15:19.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.048+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a40044f0 con 0x5578fc2a4900 2024-05-10T21:15:19.047 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:15:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac021040 con 0x5578fc2a4900 2024-05-10T21:15:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7fe3c9010b80 10 client.26441 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5578fc329da0 con 0x5578fc2a4900 2024-05-10T21:15:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a40044f0 con 0x5578fc2a4900 2024-05-10T21:15:19.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe3ac011070 con 0x5578fc2a4900 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 msgr2=0x7f7f500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 0x7f7f500b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7f64003fd0 tx=0x7f7f64004ad0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 msgr2=0x7f7f6c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.049+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c1a17f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f580098e0 tx=0x7f7f580057c0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 shutdown_connections 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f500b2880 0x7f7f500b4d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c10a7c0 0x7f7f6c1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f6c10f6c0 0x7f7f6c1a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 --2- 172.21.15.50:0/3310645225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f6c10b110 0x7f7f6c1a17f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 >> 172.21.15.50:0/3310645225 conn(0x7f7f6c076b00 msgr2=0x7f7f6c108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:19.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 10 client.26441 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 20 client.26441 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc32a020 0x5578fc32c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 4 client.26441 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f0125e47", "release": true, "await": true}] 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f0125e47", "release": true, "await": true}) -- 0x5578fc2c17b0 con 0x5578fc32a020 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.050+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc32a020 0x5578fc32c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.051+0000 7fe3a1600700 10 client.26441 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.051+0000 7fe3b3400700 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc32a020 0x5578fc32c440 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745b437bd5ebb571 server_cookie=c9d7a8d933c13941 in_seq=0 out_seq=0 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.051+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 shutdown_connections 2024-05-10T21:15:19.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.051+0000 7f7f6b200700 1 -- 172.21.15.50:0/3310645225 wait complete. 2024-05-10T21:15:19.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3a1600700 1 -- 172.21.15.50:0/1243263242 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: -1 ) ==== 8+0+590 (crc 0 0 0) 0x5578fc2c17b0 con 0x5578fc32a020 2024-05-10T21:15:19.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3a1600700 10 client.26441 handle_command_reply: tid=0 2024-05-10T21:15:19.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 client.26441 shutdown 2024-05-10T21:15:19.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 msgr2=0x5578fc2b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:19.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2b3100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009900 tx=0x7fe3ac0077f0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 shutdown_connections 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5578fc32a020 0x5578fc32c440 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3a1600700 0 client.26441 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578fc2a4900 0x5578fc2b3100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578fc126f40 0x5578fc2b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 --2- 172.21.15.50:0/1243263242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578fc2a44f0 0x5578fc2b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.052+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 >> 172.21.15.50:0/1243263242 conn(0x5578fc16c070 msgr2=0x5578fc2a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:19.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.053+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 shutdown_connections 2024-05-10T21:15:19.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.053+0000 7fe3c9010b80 1 -- 172.21.15.50:0/1243263242 wait complete. 2024-05-10T21:15:19.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:19.053+0000 7fe3c9010b80 20 client.26441 trim_cache size 0 max 16384 2024-05-10T21:15:19.055 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:15:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 33, 2024-05-10T21:15:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 15, 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": { 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "f0125e47": { 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "version": 15, 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "age_ref": 0.7, 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "EXPIRED", 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 0.0 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:15:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "timeout": 60.0, 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "expiration": 85.2, 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "members": { 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "file:/": { 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "excluded": false, 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "state": { 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "QUIESCED", 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "age": 85.2 2024-05-10T21:15:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:15:19.058 INFO:teuthology.orchestra.run.smithi050.stderr:Error EPERM: 2024-05-10T21:15:19.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:18 smithi092 ceph-mon[34415]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:15:19.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:18 smithi142 ceph-mon[34846]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:15:19.407 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:18 smithi050 ceph-mon[29546]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T21:15:20.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:15:20.475 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't release set 'f0125e47' with rc: 1 (EPERM), stdout: { "epoch": 33, "leader": 24539, "set_version": 15, "sets": { "f0125e47": { "version": 15, "age_ref": 0.7, "state": { "name": "EXPIRED", "age": 0.0 }, "timeout": 60.0, "expiration": 85.2, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCED", "age": 85.2 } } } } } } 2024-05-10T21:15:20.475 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 354, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 330, in do_quiesce raise RuntimeError(f"Error releasing set '{set_id}': {rcinfo(rc)}") RuntimeError: Error releasing set 'f0125e47': 1 (EPERM) 2024-05-10T21:15:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:20 smithi050 ceph-mon[29546]: pgmap v655: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-10T21:15:21.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:20 smithi092 ceph-mon[34415]: pgmap v655: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-10T21:15:21.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:20 smithi142 ceph-mon[34846]: pgmap v655: 97 pgs: 97 active+clean; 288 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-10T21:15:21.399 INFO:tasks.workunit.client.0.smithi050.stdout: 72 23204 98 16431 194 5291 113 2024-05-10T21:15:21.819 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:15:21.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:15:22.582 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:15:23.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:23 smithi050 ceph-mon[29546]: pgmap v656: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 190 KiB/s wr, 99 op/s 2024-05-10T21:15:23.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:23 smithi092 ceph-mon[34415]: pgmap v656: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 190 KiB/s wr, 99 op/s 2024-05-10T21:15:23.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:23 smithi142 ceph-mon[34846]: pgmap v656: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 190 KiB/s wr, 99 op/s 2024-05-10T21:15:25.088 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:25 smithi050 ceph-mon[29546]: pgmap v657: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 210 KiB/s wr, 220 op/s 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 -- 172.21.15.50:0/3966119146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c10b8a0 msgr2=0x7f118c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 --2- 172.21.15.50:0/3966119146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c10b8a0 0x7f118c113200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1184009fc0 tx=0x7f118400b330 comp rx=0 tx=0).stop 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 -- 172.21.15.50:0/3966119146 shutdown_connections 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 --2- 172.21.15.50:0/3966119146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c106620 0x7f118c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 --2- 172.21.15.50:0/3966119146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c10b8a0 0x7f118c113200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 --2- 172.21.15.50:0/3966119146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10af50 0x7f118c10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.289+0000 7f1191800700 1 -- 172.21.15.50:0/3966119146 >> 172.21.15.50:0/3966119146 conn(0x7f118c102030 msgr2=0x7f118c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.290+0000 7f1191800700 1 -- 172.21.15.50:0/3966119146 shutdown_connections 2024-05-10T21:15:25.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.290+0000 7f1191800700 1 -- 172.21.15.50:0/3966119146 wait complete. 2024-05-10T21:15:25.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.291+0000 7f1191800700 1 Processor -- start 2024-05-10T21:15:25.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.291+0000 7f1191800700 1 -- start start 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c10af50 0x7f118c112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10b8a0 0x7f118c1a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c078de0 con 0x7f118c10b8a0 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f118c078c60 con 0x7f118c10af50 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f1191800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f118c078f60 con 0x7f118c106620 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.292+0000 7f118be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c10af50 0x7f118c112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35448/0 (socket says 172.21.15.50:35448) 2024-05-10T21:15:25.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 -- 172.21.15.50:0/187979722 learned_addr learned my addr 172.21.15.50:0/187979722 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:25.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f1190e00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10b8a0 0x7f118c1a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 -- 172.21.15.50:0/187979722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c10af50 msgr2=0x7f118c112680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c10af50 0x7f118c112680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 -- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10b8a0 msgr2=0x7f118c1a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10b8a0 0x7f118c1a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.293+0000 7f118be00700 1 -- 172.21.15.50:0/187979722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c1a60f0 con 0x7f118c106620 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.294+0000 7f118be00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e9a0 tx=0x7f117c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.294+0000 7f1188c00700 1 -- 172.21.15.50:0/187979722 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c00cbd0 con 0x7f118c106620 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.294+0000 7f1188c00700 1 -- 172.21.15.50:0/187979722 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c00cd70 con 0x7f118c106620 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.294+0000 7f1188c00700 1 -- 172.21.15.50:0/187979722 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c018680 con 0x7f118c106620 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.295+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c1a63e0 con 0x7f118c106620 2024-05-10T21:15:25.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.295+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c115340 con 0x7f118c106620 2024-05-10T21:15:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.301+0000 7f1188c00700 1 -- 172.21.15.50:0/187979722 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f117c010430 con 0x7f118c106620 2024-05-10T21:15:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.302+0000 7f11a1966b80 1 Processor -- start 2024-05-10T21:15:25.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.302+0000 7f11a1966b80 1 -- start start 2024-05-10T21:15:25.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.302+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f1188c00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 0x7f11700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f11a1966b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c168b8eaa0 con 0x55c168c6d710 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f11a1966b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c168b2c7e0 con 0x55c168c6d300 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f11a1966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c168b8aa60 con 0x55c168aeeec0 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f1190e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f118be00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 0x7f11700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f1190e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35460/0 (socket says 172.21.15.50:35460) 2024-05-10T21:15:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.303+0000 7f1190e00700 1 -- 172.21.15.50:0/126554657 learned_addr learned my addr 172.21.15.50:0/126554657 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c78eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f118b400700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 -- 172.21.15.50:0/126554657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 msgr2=0x55c168c76960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 -- 172.21.15.50:0/126554657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 msgr2=0x55c168c78eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c78eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 -- 172.21.15.50:0/126554657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c168adb950 con 0x55c168aeeec0 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1188c00700 1 -- 172.21.15.50:0/187979722 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f117c0e24d0 con 0x7f118c106620 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f118b400700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.304+0000 7f1190e00700 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1184009f90 tx=0x7f1184009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baee75200ee0082f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.305+0000 7f1181600700 1 -- 172.21.15.50:0/126554657 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400b9f0 con 0x55c168aeeec0 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.305+0000 7f1181600700 1 -- 172.21.15.50:0/126554657 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118400bb90 con 0x55c168aeeec0 2024-05-10T21:15:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.305+0000 7f1181600700 1 -- 172.21.15.50:0/126554657 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118401f6d0 con 0x55c168aeeec0 2024-05-10T21:15:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 -- 172.21.15.50:0/126554657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 msgr2=0x55c168c6cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f118be00700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 0x7f11700b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a6d00 tx=0x7f117c0e5cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1184009f90 tx=0x7f1184009770 comp rx=0 tx=0).stop 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 -- 172.21.15.50:0/126554657 shutdown_connections 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c78eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c6cdc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 --2- 172.21.15.50:0/126554657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 -- 172.21.15.50:0/126554657 >> 172.21.15.50:0/126554657 conn(0x55c168b2db00 msgr2=0x55c168c67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.306+0000 7f11a1966b80 1 -- 172.21.15.50:0/126554657 shutdown_connections 2024-05-10T21:15:25.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.307+0000 7f11a1966b80 1 -- 172.21.15.50:0/126554657 wait complete. 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.307+0000 7f11a1966b80 1 -- start start 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.307+0000 7f11a1966b80 1 Processor -- start 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c76450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c168c797c0 con 0x55c168c6d710 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c168af4ba0 con 0x55c168c6d300 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c168c79940 con 0x55c168aeeec0 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.308+0000 7f11a1966b80 10 client.0 _ll_register_callbacks cb 0x55c168b22400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f1190e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c76450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f1190e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c76450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35464/0 (socket says 172.21.15.50:35464) 2024-05-10T21:15:25.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f1190e00700 1 -- 172.21.15.50:0/3387266804 learned_addr learned my addr 172.21.15.50:0/3387266804 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f1190e00700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 -- 172.21.15.50:0/3387266804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 msgr2=0x55c168c76450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c76450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 -- 172.21.15.50:0/3387266804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 msgr2=0x55c168c76ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c76ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f118b400700 1 -- 172.21.15.50:0/3387266804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168b544a0 con 0x55c168c6d300 2024-05-10T21:15:25.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.309+0000 7f1190e00700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.310+0000 7f118b400700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f118c07b050 tx=0x7f1178013ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98994d2f0ed1c4a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.310+0000 7f1176000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.310+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178004520 con 0x55c168c6d300 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f1178003680 con 0x55c168c6d300 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118400bb90 con 0x55c168c6d300 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1178003880 con 0x55c168c6d300 2024-05-10T21:15:25.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f11a1966b80 10 client.26450 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c168c84430 con 0x55c168c6d300 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.311+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118400bb90 con 0x55c168c6d300 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1178027480 con 0x55c168c6d300 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 10 client.26450 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 10 client.26450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 20 client.26450 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168c869c0 0x55c168c88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 4 client.26450 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:15:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c168c390f0 con 0x55c168c869c0 2024-05-10T21:15:25.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.312+0000 7f1190e00700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168c869c0 0x55c168c88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.314+0000 7f1190e00700 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168c869c0 0x55c168c88de0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d2586e8a0ed1d8 server_cookie=c78adb85bc7ef47 in_seq=0 out_seq=0 2024-05-10T21:15:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.314+0000 7f1176000700 10 client.26450 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:25.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.314+0000 7f1176000700 1 -- 172.21.15.50:0/3387266804 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c168c390f0 con 0x55c168c869c0 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.314+0000 7f1176000700 10 client.26450 handle_command_reply: tid=0 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.315+0000 7f11a1966b80 1 client.26450 shutdown 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.316+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 msgr2=0x55c168c76990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.316+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f118c07b050 tx=0x7f1178013ab0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 shutdown_connections 2024-05-10T21:15:25.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168c869c0 0x55c168c88de0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168c76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168aeeec0 0x55c168c76450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 --2- 172.21.15.50:0/3387266804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c76990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 >> 172.21.15.50:0/3387266804 conn(0x55c168b2db00 msgr2=0x55c168c6b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f1176000700 0 client.26450 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:25.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 shutdown_connections 2024-05-10T21:15:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 1 -- 172.21.15.50:0/3387266804 wait complete. 2024-05-10T21:15:25.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.317+0000 7f11a1966b80 20 client.26450 trim_cache size 0 max 16384 2024-05-10T21:15:25.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.363+0000 7f11a1966b80 1 Processor -- start 2024-05-10T21:15:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 -- start start 2024-05-10T21:15:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168aeeec0 0x55c168c76d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d710 0x55c168c71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1c0070 con 0x55c168aeeec0 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f118c11c6e0 con 0x55c168c6d300 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f11a1966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f118c11a550 con 0x55c168c6d710 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.364+0000 7f118b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33032/0 (socket says 172.21.15.50:33032) 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 -- 172.21.15.50:0/2754984807 learned_addr learned my addr 172.21.15.50:0/2754984807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f1190e00700 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168aeeec0 0x55c168c76d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 -- 172.21.15.50:0/2754984807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d710 msgr2=0x55c168c71bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d710 0x55c168c71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 -- 172.21.15.50:0/2754984807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168aeeec0 msgr2=0x55c168c76d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168aeeec0 0x55c168c76d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.365+0000 7f118b400700 1 -- 172.21.15.50:0/2754984807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c168c6c3a0 con 0x55c168c6d300 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.366+0000 7f118b400700 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1178027f60 tx=0x7f1178003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1a05569c42d5a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.366+0000 7f1180c00700 1 -- 172.21.15.50:0/2754984807 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11780053c0 con 0x55c168c6d300 2024-05-10T21:15:25.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.366+0000 7f1180c00700 1 -- 172.21.15.50:0/2754984807 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1178027a90 con 0x55c168c6d300 2024-05-10T21:15:25.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.366+0000 7f1180c00700 1 -- 172.21.15.50:0/2754984807 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178011a50 con 0x55c168c6d300 2024-05-10T21:15:25.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 -- 172.21.15.50:0/2754984807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 msgr2=0x55c168c716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1178027f60 tx=0x7f1178003680 comp rx=0 tx=0).stop 2024-05-10T21:15:25.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:25 smithi092 ceph-mon[34415]: pgmap v657: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 210 KiB/s wr, 220 op/s 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 -- 172.21.15.50:0/2754984807 shutdown_connections 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d710 0x55c168c71bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168aeeec0 0x55c168c76d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 --2- 172.21.15.50:0/2754984807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168c6d300 0x55c168c716b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 -- 172.21.15.50:0/2754984807 >> 172.21.15.50:0/2754984807 conn(0x55c168b2db00 msgr2=0x55c168c66f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.368+0000 7f11a1966b80 1 -- 172.21.15.50:0/2754984807 shutdown_connections 2024-05-10T21:15:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.369+0000 7f11a1966b80 1 -- 172.21.15.50:0/2754984807 wait complete. 2024-05-10T21:15:25.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.369+0000 7f11a1966b80 1 -- start start 2024-05-10T21:15:25.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 Processor -- start 2024-05-10T21:15:25.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d300 0x55c168c7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168cf0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c11aba0 con 0x55c168c6d710 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f118c11c010 con 0x55c168aeeec0 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.370+0000 7f11a1966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f118c11b270 con 0x55c168c6d300 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f11a1966b80 10 client.0 _ll_register_callbacks cb 0x55c168b22400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f1190e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f118b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d300 0x55c168c7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f1190e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33040/0 (socket says 172.21.15.50:33040) 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f1190e00700 1 -- 172.21.15.50:0/910528535 learned_addr learned my addr 172.21.15.50:0/910528535 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:25.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.371+0000 7f1190e00700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168cf0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 -- 172.21.15.50:0/910528535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d300 msgr2=0x55c168c7b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d300 0x55c168c7b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 -- 172.21.15.50:0/910528535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 msgr2=0x55c168cf0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168cf0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 -- 172.21.15.50:0/910528535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168cf06b0 con 0x55c168aeeec0 2024-05-10T21:15:25.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1190e00700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1184007dd0 tx=0x7f1184021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc83f615c515d70b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.372+0000 7f1176000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400bad0 con 0x55c168aeeec0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f118400f040 con 0x55c168aeeec0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1178027a90 con 0x55c168aeeec0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f118400bc70 con 0x55c168aeeec0 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f11a1966b80 10 client.26456 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:25.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.373+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c168cf1950 con 0x55c168aeeec0 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.374+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1178027a90 con 0x55c168aeeec0 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.374+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1184011070 con 0x55c168aeeec0 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 20 client.26456 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168cf1c30 0x55c168cf4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 4 client.26456 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c168b9c480 con 0x55c168cf1c30 2024-05-10T21:15:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.375+0000 7f118b400700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168cf1c30 0x55c168cf4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:25.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.376+0000 7f118b400700 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168cf1c30 0x55c168cf4050 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf4ebf3f38de623 server_cookie=bfaef575f03127eb in_seq=0 out_seq=0 2024-05-10T21:15:25.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.376+0000 7f1176000700 10 client.26456 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:25.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.377+0000 7f1176000700 1 -- 172.21.15.50:0/910528535 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c168b9c480 con 0x55c168cf1c30 2024-05-10T21:15:25.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.377+0000 7f1176000700 10 client.26456 handle_command_reply: tid=0 2024-05-10T21:15:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.377+0000 7f11a1966b80 1 client.26456 shutdown 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.377+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 msgr2=0x55c168c7af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.377+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1184007dd0 tx=0x7f1184021040 comp rx=0 tx=0).stop 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 shutdown_connections 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c168cf1c30 0x55c168cf4050 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c168aeeec0 0x55c168c7af50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f1176000700 0 client.26456 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c168c6d710 0x55c168cf0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 --2- 172.21.15.50:0/910528535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c168c6d300 0x55c168c7b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 >> 172.21.15.50:0/910528535 conn(0x55c168b2db00 msgr2=0x55c168c67940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.378+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 shutdown_connections 2024-05-10T21:15:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.379+0000 7f11a1966b80 1 -- 172.21.15.50:0/910528535 wait complete. 2024-05-10T21:15:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.379+0000 7f11a1966b80 20 client.26456 trim_cache size 0 max 16384 2024-05-10T21:15:25.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:25 smithi142 ceph-mon[34846]: pgmap v657: 97 pgs: 97 active+clean; 289 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 210 KiB/s wr, 220 op/s 2024-05-10T21:15:25.378 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:15:25.378 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:15:25.378 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "26d55de3-7dd8-42f5-862d-52bac98c1695", 2024-05-10T21:15:25.378 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:15:25.378 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 msgr2=0x7f11700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 0x7f11700b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a6d00 tx=0x7f117c0e5cc0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 msgr2=0x7f118c112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e9a0 tx=0x7f117c00ecb0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 shutdown_connections 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f11700b2880 0x7f11700b4d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c10b8a0 0x7f118c1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f118c106620 0x7f118c112120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 --2- 172.21.15.50:0/187979722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f118c10af50 0x7f118c112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:25.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.380+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 >> 172.21.15.50:0/187979722 conn(0x7f118c102030 msgr2=0x7f118c109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.382+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 shutdown_connections 2024-05-10T21:15:25.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:25.382+0000 7f1191800700 1 -- 172.21.15.50:0/187979722 wait complete. 2024-05-10T21:15:26.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:15:26.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:26.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:26 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:26.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:26 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:26.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:26 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:26.833 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:15:27.266 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.267 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:27 smithi092 ceph-mon[34415]: pgmap v658: 97 pgs: 97 active+clean; 297 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 918 KiB/s wr, 392 op/s 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.321 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:27 smithi050 ceph-mon[29546]: pgmap v658: 97 pgs: 97 active+clean; 297 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 918 KiB/s wr, 392 op/s 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:27.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:27 smithi142 ceph-mon[34846]: pgmap v658: 97 pgs: 97 active+clean; 297 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 918 KiB/s wr, 392 op/s 2024-05-10T21:15:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:29 smithi050 ceph-mon[29546]: pgmap v659: 97 pgs: 97 active+clean; 304 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-05-10T21:15:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:29 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:15:29.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:29 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:29 smithi092 ceph-mon[34415]: pgmap v659: 97 pgs: 97 active+clean; 304 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-05-10T21:15:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:29 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:15:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:29 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:29 smithi142 ceph-mon[34846]: pgmap v659: 97 pgs: 97 active+clean; 304 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.5 MiB/s wr, 520 op/s 2024-05-10T21:15:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:29 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:15:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:29 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.993+0000 7fbbf6200700 1 -- 172.21.15.50:0/2798626342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 msgr2=0x7fbbf01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.993+0000 7fbbf6200700 1 --2- 172.21.15.50:0/2798626342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf01038d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00099c0 tx=0x7fbbe0009cd0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 -- 172.21.15.50:0/2798626342 shutdown_connections 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 --2- 172.21.15.50:0/2798626342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0108740 0x7fbbf01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 --2- 172.21.15.50:0/2798626342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf01038d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 --2- 172.21.15.50:0/2798626342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0107df0 0x7fbbf0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 -- 172.21.15.50:0/2798626342 >> 172.21.15.50:0/2798626342 conn(0x7fbbf00fedc0 msgr2=0x7fbbf01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 -- 172.21.15.50:0/2798626342 shutdown_connections 2024-05-10T21:15:29.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.994+0000 7fbbf6200700 1 -- 172.21.15.50:0/2798626342 wait complete. 2024-05-10T21:15:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.995+0000 7fbbf6200700 1 Processor -- start 2024-05-10T21:15:29.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.996+0000 7fbbf6200700 1 -- start start 2024-05-10T21:15:29.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.996+0000 7fbbf6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:29.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.996+0000 7fbbf6200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0107df0 0x7fbbf019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbf6200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0108740 0x7fbbf019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbf6200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01133c0 con 0x7fbbf0108740 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbf6200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0113240 con 0x7fbbf0107df0 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbf6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0113540 con 0x7fbbf01034c0 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35522/0 (socket says 172.21.15.50:35522) 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 -- 172.21.15.50:0/1966719386 learned_addr learned my addr 172.21.15.50:0/1966719386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:29.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbeea00700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0107df0 0x7fbbf019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:29.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbefe00700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0108740 0x7fbbf019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 -- 172.21.15.50:0/1966719386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0107df0 msgr2=0x7fbbf019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0107df0 0x7fbbf019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 -- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0108740 msgr2=0x7fbbf019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0108740 0x7fbbf019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.997+0000 7fbbef400700 1 -- 172.21.15.50:0/1966719386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf019fcb0 con 0x7fbbf01034c0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.998+0000 7fbbef400700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005910 tx=0x7fbbe0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.998+0000 7fbbe7e00700 1 -- 172.21.15.50:0/1966719386 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0026070 con 0x7fbbf01034c0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.998+0000 7fbbe7e00700 1 -- 172.21.15.50:0/1966719386 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0003c70 con 0x7fbbf01034c0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.998+0000 7fbbe7e00700 1 -- 172.21.15.50:0/1966719386 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000fa40 con 0x7fbbf01034c0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.998+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf019ff40 con 0x7fbbf01034c0 2024-05-10T21:15:29.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:29.999+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a03d0 con 0x7fbbf01034c0 2024-05-10T21:15:30.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.005+0000 7fbbe7e00700 1 -- 172.21.15.50:0/1966719386 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbbe0029070 con 0x7fbbf01034c0 2024-05-10T21:15:30.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.006+0000 7fbc04d86b80 1 Processor -- start 2024-05-10T21:15:30.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.006+0000 7fbc04d86b80 1 -- start start 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb306710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb307060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb31b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb228a20 con 0x55f8bb319370 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb1c6760 con 0x55f8bb306c50 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb2249e0 con 0x55f8bb188e40 2024-05-10T21:15:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbbef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb307060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbbef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb307060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33066/0 (socket says 172.21.15.50:33066) 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbbef400700 1 -- 172.21.15.50:0/2967159478 learned_addr learned my addr 172.21.15.50:0/2967159478 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.008+0000 7fbbeea00700 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb31b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbefe00700 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb306710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbeea00700 1 -- 172.21.15.50:0/2967159478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 msgr2=0x55f8bb306710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbeea00700 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb306710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbeea00700 1 -- 172.21.15.50:0/2967159478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 msgr2=0x55f8bb307060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbeea00700 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb307060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.009+0000 7fbbeea00700 1 -- 172.21.15.50:0/2967159478 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8bb1758d0 con 0x55f8bb319370 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.010+0000 7fbbeea00700 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb31b790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0076ab0 tx=0x7fbbdc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2754fa755abaf076 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.010+0000 7fbbe7e00700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 0x7fbbd40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.010+0000 7fbbe4c00700 1 -- 172.21.15.50:0/2967159478 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc011c80 con 0x55f8bb319370 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.010+0000 7fbbe7e00700 1 -- 172.21.15.50:0/1966719386 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fbbe00e2b10 con 0x7fbbf01034c0 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.011+0000 7fbbe4c00700 1 -- 172.21.15.50:0/2967159478 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc011e20 con 0x55f8bb319370 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.011+0000 7fbbe4c00700 1 -- 172.21.15.50:0/2967159478 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc005280 con 0x55f8bb319370 2024-05-10T21:15:30.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.011+0000 7fbbefe00700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 0x7fbbd40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2967159478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 msgr2=0x55f8bb31b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb31b790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0076ab0 tx=0x7fbbdc009f60 comp rx=0 tx=0).stop 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbbefe00700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 0x7fbbd40b4e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f8bb307950 tx=0x7fbbe800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2967159478 shutdown_connections 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb31b790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb306710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2967159478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb307060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.013+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2967159478 >> 172.21.15.50:0/2967159478 conn(0x55f8bb1c7a80 msgr2=0x55f8bb301790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:30.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.014+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2967159478 shutdown_connections 2024-05-10T21:15:30.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.014+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2967159478 wait complete. 2024-05-10T21:15:30.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.014+0000 7fbc04d86b80 1 -- start start 2024-05-10T21:15:30.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.014+0000 7fbc04d86b80 1 Processor -- start 2024-05-10T21:15:30.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb188e40 0x55f8bb316350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb316890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb31beb0 con 0x55f8bb306c50 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb18eb20 con 0x55f8bb188e40 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f8bb31c030 con 0x55f8bb319370 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.015+0000 7fbc04d86b80 10 client.0 _ll_register_callbacks cb 0x55f8bb1bc380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35536/0 (socket says 172.21.15.50:35536) 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 -- 172.21.15.50:0/2767143177 learned_addr learned my addr 172.21.15.50:0/2767143177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:30.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbeea00700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb316890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 -- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb188e40 msgr2=0x55f8bb316350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T21:15:30.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb188e40 0x55f8bb316350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 -- 172.21.15.50:0/2767143177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 msgr2=0x55f8bb316890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb316890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.016+0000 7fbbefe00700 1 -- 172.21.15.50:0/2767143177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8bb1ee420 con 0x55f8bb319370 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.017+0000 7fbbefe00700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8016e00 tx=0x7fbbe8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9173f512622713b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.017+0000 7fbbd9600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:15:30.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.017+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8026070 con 0x55f8bb319370 2024-05-10T21:15:30.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.018+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fbbe802b460 con 0x55f8bb319370 2024-05-10T21:15:30.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.018+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc011e20 con 0x55f8bb319370 2024-05-10T21:15:30.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.019+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbe802b660 con 0x55f8bb319370 2024-05-10T21:15:30.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.020+0000 7fbc04d86b80 10 client.26416 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:30.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.020+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f8bb326b60 con 0x55f8bb319370 2024-05-10T21:15:30.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc011e20 con 0x55f8bb319370 2024-05-10T21:15:30.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbe8035920 con 0x55f8bb319370 2024-05-10T21:15:30.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 10 client.26416 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:30.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:30.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 20 client.26416 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:30.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb3291a0 0x55f8bb32b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 4 client.26416 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:15:30.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.021+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8bb2d3070 con 0x55f8bb3291a0 2024-05-10T21:15:30.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.023+0000 7fbbef400700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb3291a0 0x55f8bb32b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.026+0000 7fbbd9600700 10 client.26416 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.026+0000 7fbbef400700 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb3291a0 0x55f8bb32b650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfbd4b6bea78e78 server_cookie=1348538115a37bb8 in_seq=0 out_seq=0 2024-05-10T21:15:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.028+0000 7fbbd9600700 1 -- 172.21.15.50:0/2767143177 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8bb2d3070 con 0x55f8bb3291a0 2024-05-10T21:15:30.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.028+0000 7fbbd9600700 10 client.26416 handle_command_reply: tid=0 2024-05-10T21:15:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.028+0000 7fbc04d86b80 1 client.26416 shutdown 2024-05-10T21:15:30.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.028+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 msgr2=0x55f8bb316dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.028+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8016e00 tx=0x7fbbe8004b40 comp rx=0 tx=0).stop 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 shutdown_connections 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb3291a0 0x55f8bb32b650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb188e40 0x55f8bb316350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb319370 0x55f8bb316dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/2767143177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb316890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 >> 172.21.15.50:0/2767143177 conn(0x55f8bb1c7a80 msgr2=0x55f8bb319780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbbd9600700 0 client.26416 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 shutdown_connections 2024-05-10T21:15:30.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.029+0000 7fbc04d86b80 1 -- 172.21.15.50:0/2767143177 wait complete. 2024-05-10T21:15:30.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.030+0000 7fbc04d86b80 20 client.26416 trim_cache size 0 max 16384 2024-05-10T21:15:30.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.077+0000 7fbc04d86b80 1 Processor -- start 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.077+0000 7fbc04d86b80 1 -- start start 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb3015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb324f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb330e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01af060 con 0x55f8bb319370 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ac890 con 0x55f8bb306c50 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.078+0000 7fbc04d86b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01aa700 con 0x55f8bb188e40 2024-05-10T21:15:30.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.079+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb3015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.079+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb3015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35544/0 (socket says 172.21.15.50:35544) 2024-05-10T21:15:30.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.079+0000 7fbbef400700 1 -- 172.21.15.50:0/158449049 learned_addr learned my addr 172.21.15.50:0/158449049 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:30.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.080+0000 7fbbeea00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb324f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.080+0000 7fbbefe00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb330e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.082+0000 7fbbeea00700 1 -- 172.21.15.50:0/158449049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 msgr2=0x55f8bb3015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.082+0000 7fbbeea00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb3015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.082+0000 7fbbeea00700 1 -- 172.21.15.50:0/158449049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 msgr2=0x55f8bb330e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.082+0000 7fbbeea00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb330e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.082+0000 7fbbeea00700 1 -- 172.21.15.50:0/158449049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8bb317360 con 0x55f8bb306c50 2024-05-10T21:15:30.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.083+0000 7fbbefe00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb330e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:30.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.083+0000 7fbbeea00700 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb324f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc004de0 tx=0x7fbbdc009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670cda2cdd5d884f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:30.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.085+0000 7fbbdbe00700 1 -- 172.21.15.50:0/158449049 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc027020 con 0x55f8bb306c50 2024-05-10T21:15:30.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.085+0000 7fbbdbe00700 1 -- 172.21.15.50:0/158449049 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc011b80 con 0x55f8bb306c50 2024-05-10T21:15:30.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.085+0000 7fbbdbe00700 1 -- 172.21.15.50:0/158449049 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc005210 con 0x55f8bb306c50 2024-05-10T21:15:30.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.087+0000 7fbc04d86b80 1 -- 172.21.15.50:0/158449049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 msgr2=0x55f8bb324f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.087+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb324f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc004de0 tx=0x7fbbdc009710 comp rx=0 tx=0).stop 2024-05-10T21:15:30.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.088+0000 7fbc04d86b80 1 -- 172.21.15.50:0/158449049 shutdown_connections 2024-05-10T21:15:30.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.088+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb319370 0x55f8bb330e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.088+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb3015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.088+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/158449049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb306c50 0x55f8bb324f40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.088+0000 7fbc04d86b80 1 -- 172.21.15.50:0/158449049 >> 172.21.15.50:0/158449049 conn(0x55f8bb1c7a80 msgr2=0x55f8bb300e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:30.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.089+0000 7fbc04d86b80 1 -- 172.21.15.50:0/158449049 shutdown_connections 2024-05-10T21:15:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.089+0000 7fbc04d86b80 1 -- 172.21.15.50:0/158449049 wait complete. 2024-05-10T21:15:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.090+0000 7fbc04d86b80 1 -- start start 2024-05-10T21:15:30.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.090+0000 7fbc04d86b80 1 Processor -- start 2024-05-10T21:15:30.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.092+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb319f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.092+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb31a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.092+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb319f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.092+0000 7fbbef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb319f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35558/0 (socket says 172.21.15.50:35558) 2024-05-10T21:15:30.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.093+0000 7fbc04d86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.093+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.094+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33088/0 (socket says 172.21.15.50:33088) 2024-05-10T21:15:30.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.094+0000 7fbbefe00700 1 -- 172.21.15.50:0/4124658339 learned_addr learned my addr 172.21.15.50:0/4124658339 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:30.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.094+0000 7fbbeea00700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb31a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.095+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01aad50 con 0x55f8bb306c50 2024-05-10T21:15:30.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.095+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ac1c0 con 0x55f8bb319370 2024-05-10T21:15:30.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.095+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ab420 con 0x55f8bb188e40 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbefe00700 1 -- 172.21.15.50:0/4124658339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 msgr2=0x55f8bb319f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbefe00700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb319f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbefe00700 1 -- 172.21.15.50:0/4124658339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 msgr2=0x55f8bb31a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbefe00700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb31a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbefe00700 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8bb3929c0 con 0x55f8bb319370 2024-05-10T21:15:30.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.096+0000 7fbbeea00700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb31a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:15:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.097+0000 7fbbefe00700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8016dd0 tx=0x7fbbe802b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174a05fc9c764a07 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.097+0000 7fbbd9600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.097+0000 7fbbd9600700 1 -- 172.21.15.50:0/4124658339 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe803a040 con 0x55f8bb319370 2024-05-10T21:15:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.097+0000 7fbbd9600700 1 -- 172.21.15.50:0/4124658339 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fbbe802b800 con 0x55f8bb319370 2024-05-10T21:15:30.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.098+0000 7fbc04d86b80 10 client.0 _ll_register_callbacks cb 0x55f8bb1bc380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.098+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc011b80 con 0x55f8bb319370 2024-05-10T21:15:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.098+0000 7fbbd9600700 1 -- 172.21.15.50:0/4124658339 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbe8026040 con 0x55f8bb319370 2024-05-10T21:15:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.098+0000 7fbc04d86b80 10 client.26471 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:30.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.098+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55f8bb393840 con 0x55f8bb319370 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbbd9600700 1 -- 172.21.15.50:0/4124658339 <== mon.2 v2:172.21.15.92:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbe8048030 con 0x55f8bb319370 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 10 client.26471 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 10 client.26471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 20 client.26471 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb393f00 0x55f8bb396410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 4 client.26471 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:15:30.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8bb236400 con 0x55f8bb393f00 2024-05-10T21:15:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.099+0000 7fbbef400700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb393f00 0x55f8bb396410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.100+0000 7fbbef400700 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb393f00 0x55f8bb396410 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f290289262727ccb server_cookie=c08e34d1ee25cf0f in_seq=0 out_seq=0 2024-05-10T21:15:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.100+0000 7fbbd9600700 10 client.26471 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.101+0000 7fbbd9600700 1 -- 172.21.15.50:0/4124658339 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f8bb236400 con 0x55f8bb393f00 2024-05-10T21:15:30.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.101+0000 7fbbd9600700 10 client.26471 handle_command_reply: tid=0 2024-05-10T21:15:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.101+0000 7fbc04d86b80 1 client.26471 shutdown 2024-05-10T21:15:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.101+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 msgr2=0x55f8bb31aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.101+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8016dd0 tx=0x7fbbe802b460 comp rx=0 tx=0).stop 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 shutdown_connections 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f8bb393f00 0x55f8bb396410 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbbd9600700 0 client.26471 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8bb319370 0x55f8bb31aa10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8bb188e40 0x55f8bb319f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 --2- 172.21.15.50:0/4124658339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8bb306c50 0x55f8bb31a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 >> 172.21.15.50:0/4124658339 conn(0x55f8bb1c7a80 msgr2=0x55f8bb315c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:30.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.102+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 shutdown_connections 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.103+0000 7fbc04d86b80 1 -- 172.21.15.50:0/4124658339 wait complete. 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.103+0000 7fbc04d86b80 20 client.26471 trim_cache size 0 max 16384 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stdout: "26d55de3-7dd8-42f5-862d-52bac98c1695": { 2024-05-10T21:15:30.100 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "26d55de3-7dd8-42f5-862d-52bac98c1695", 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:15:30.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 msgr2=0x7fbbd40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 0x7fbbd40b4e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f8bb307950 tx=0x7fbbe800b040 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 msgr2=0x7fbbf019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005910 tx=0x7fbbe0004270 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 shutdown_connections 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbbd40b2900 0x7fbbd40b4e10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbf0108740 0x7fbbf019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf01034c0 0x7fbbf019ade0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 --2- 172.21.15.50:0/1966719386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0107df0 0x7fbbf019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.104+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 >> 172.21.15.50:0/1966719386 conn(0x7fbbf00fedc0 msgr2=0x7fbbf01063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.105+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 shutdown_connections 2024-05-10T21:15:30.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:15:30.105+0000 7fbbf6200700 1 -- 172.21.15.50:0/1966719386 wait complete. 2024-05-10T21:15:31.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:31 smithi050 ceph-mon[29546]: pgmap v660: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 589 op/s 2024-05-10T21:15:31.281 INFO:tasks.cephfs.filesystem:scrub status for tag:26d55de3-7dd8-42f5-862d-52bac98c1695 - {'path': '/', 'tag': '26d55de3-7dd8-42f5-862d-52bac98c1695', 'options': 'recursive,force'} 2024-05-10T21:15:31.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:31 smithi092 ceph-mon[34415]: pgmap v660: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 589 op/s 2024-05-10T21:15:31.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:31 smithi142 ceph-mon[34846]: pgmap v660: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 589 op/s 2024-05-10T21:15:31.399 INFO:tasks.workunit.client.0.smithi050.stdout: 75 116183 256 77239 96 34513 331 2024-05-10T21:15:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 -- 172.21.15.50:0/4186106973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 msgr2=0x7f09fc1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 --2- 172.21.15.50:0/4186106973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc1065e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009900 tx=0x7f09ec02a410 comp rx=0 tx=0).stop 2024-05-10T21:15:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 -- 172.21.15.50:0/4186106973 shutdown_connections 2024-05-10T21:15:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 --2- 172.21.15.50:0/4186106973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc109720 0x7f09fc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 --2- 172.21.15.50:0/4186106973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc106bb0 0x7f09fc109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 --2- 172.21.15.50:0/4186106973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc1065e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 -- 172.21.15.50:0/4186106973 >> 172.21.15.50:0/4186106973 conn(0x7f09fc0fdb60 msgr2=0x7f09fc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 -- 172.21.15.50:0/4186106973 shutdown_connections 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.294+0000 7f09fbe00640 1 -- 172.21.15.50:0/4186106973 wait complete. 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.296+0000 7f09fbe00640 1 Processor -- start 2024-05-10T21:15:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.296+0000 7f09fbe00640 1 -- start start 2024-05-10T21:15:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc106bb0 0x7f09fc1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc10f580 con 0x7f09fc1041c0 2024-05-10T21:15:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc10f400 con 0x7f09fc109720 2024-05-10T21:15:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc10f700 con 0x7f09fc106bb0 2024-05-10T21:15:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fb400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fb400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33114/0 (socket says 172.21.15.50:33114) 2024-05-10T21:15:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fb400640 1 -- 172.21.15.50:0/2215553900 learned_addr learned my addr 172.21.15.50:0/2215553900 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09fa000640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc106bb0 0x7f09fc1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.297+0000 7f09faa00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 -- 172.21.15.50:0/2215553900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc106bb0 msgr2=0x7f09fc1020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc106bb0 0x7f09fc1020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 -- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 msgr2=0x7f09fc103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 -- 172.21.15.50:0/2215553900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc102d40 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09fb400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09f400efd0 tx=0x7f09f400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09f3400640 1 -- 172.21.15.50:0/2215553900 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4003b70 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09f3400640 1 -- 172.21.15.50:0/2215553900 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f4003d10 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.298+0000 7f09f3400640 1 -- 172.21.15.50:0/2215553900 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40108a0 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.299+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc1a4220 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.299+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a4690 con 0x7f09fc109720 2024-05-10T21:15:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.303+0000 7f09f3400640 1 -- 172.21.15.50:0/2215553900 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f09f4004020 con 0x7f09fc109720 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.303+0000 7f0a08c23740 1 Processor -- start 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 -- start start 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 0x564c377b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c3771a550 con 0x564c373eb2f0 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564c376bfbc0 con 0x564c373f34d0 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.304+0000 7f0a08c23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c37716510 con 0x564c374141b0 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35582/0 (socket says 172.21.15.50:35582) 2024-05-10T21:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 -- 172.21.15.50:0/86584798 learned_addr learned my addr 172.21.15.50:0/86584798 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09faa00640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 0x564c377b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09f3400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 0x7f09e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09f3400640 1 -- 172.21.15.50:0/2215553900 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f09f40e25e0 con 0x7f09fc109720 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 -- 172.21.15.50:0/86584798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 msgr2=0x564c377bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 -- 172.21.15.50:0/86584798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 msgr2=0x564c377b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 0x564c377b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fa000640 1 -- 172.21.15.50:0/86584798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3742c210 con 0x564c374141b0 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09fb400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 0x7f09e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.305+0000 7f09faa00640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 0x564c377b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.306+0000 7f09fa000640 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f09e8013820 tx=0x7f09e8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28cd7a4570f795be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.306+0000 7f09e7e00640 1 -- 172.21.15.50:0/86584798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8013da0 con 0x564c374141b0 2024-05-10T21:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.306+0000 7f09e7e00640 1 -- 172.21.15.50:0/86584798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e8013170 con 0x564c374141b0 2024-05-10T21:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.306+0000 7f09e7e00640 1 -- 172.21.15.50:0/86584798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8010890 con 0x564c374141b0 2024-05-10T21:15:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 -- 172.21.15.50:0/86584798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 msgr2=0x564c377b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f09fb400640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 0x7f09e00b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09f401a040 tx=0x7f09f4009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f09e8013820 tx=0x7f09e8005f90 comp rx=0 tx=0).stop 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 -- 172.21.15.50:0/86584798 shutdown_connections 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373eb2f0 0x564c377b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 --2- 172.21.15.50:0/86584798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377b6300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 -- 172.21.15.50:0/86584798 >> 172.21.15.50:0/86584798 conn(0x564c37408300 msgr2=0x564c377b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.307+0000 7f0a08c23740 1 -- 172.21.15.50:0/86584798 shutdown_connections 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.308+0000 7f0a08c23740 1 -- 172.21.15.50:0/86584798 wait complete. 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.308+0000 7f0a08c23740 1 -- start start 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 Processor -- start 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 0x564c377bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c377c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c377be1d0 con 0x564c373f34d0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564c376d3660 con 0x564c373eb2f0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c377be350 con 0x564c374141b0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f0a08c23740 10 client.0 _ll_register_callbacks cb 0x564c3769d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f09fa000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 0x564c377bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f09faa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c377c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f09fa000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 0x564c377bfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33130/0 (socket says 172.21.15.50:33130) 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.309+0000 7f09fa000640 1 -- 172.21.15.50:0/2935852139 learned_addr learned my addr 172.21.15.50:0/2935852139 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09fa000640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 -- 172.21.15.50:0/2935852139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 msgr2=0x564c377c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 -- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 msgr2=0x564c377bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 0x564c377bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 -- 172.21.15.50:0/2935852139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c374f8620 con 0x564c373f34d0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09faa00640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c377c0310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09ec02aa00 tx=0x7f09ec004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbc75887e9f948e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.310+0000 7f09e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec035040 con 0x564c373f34d0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f09ec005230 con 0x564c373f34d0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e8013170 con 0x564c373f34d0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f09ec04d050 con 0x564c373f34d0 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f0a08c23740 10 client.16344 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c377ca210 con 0x564c373f34d0 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.311+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e8013170 con 0x564c373f34d0 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f09ec003740 con 0x564c373f34d0 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 10 client.16344 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 20 client.16344 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373d2d80 0x564c377cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 4 client.16344 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c37728160 con 0x564c373d2d80 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.312+0000 7f09fa000640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373d2d80 0x564c377cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.313+0000 7f09fa000640 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373d2d80 0x564c377cd930 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773db22f8a0592c9 server_cookie=cadf5ec2f2128df4 in_seq=0 out_seq=0 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.313+0000 7f09e4c00640 10 client.16344 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f09e4c00640 1 -- 172.21.15.50:0/2935852139 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c37728160 con 0x564c373d2d80 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f09e4c00640 10 client.16344 handle_command_reply: tid=0 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 client.16344 shutdown 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 msgr2=0x564c377c0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c377c0310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09ec02aa00 tx=0x7f09ec004550 comp rx=0 tx=0).stop 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 shutdown_connections 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373d2d80 0x564c377cd930 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c374141b0 0x564c377c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c377c0310 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 --2- 172.21.15.50:0/2935852139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373eb2f0 0x564c377bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f09e4c00640 0 client.16344 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 >> 172.21.15.50:0/2935852139 conn(0x564c37408300 msgr2=0x564c377b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 shutdown_connections 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.314+0000 7f0a08c23740 1 -- 172.21.15.50:0/2935852139 wait complete. 2024-05-10T21:15:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.315+0000 7f0a08c23740 20 client.16344 trim_cache size 0 max 16384 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 Processor -- start 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 -- start start 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c377d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 0x564c377d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1afa80 con 0x564c374141b0 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1ad2f0 con 0x564c373f34d0 2024-05-10T21:15:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.335+0000 7f0a08c23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1ab180 con 0x564c373eb2f0 2024-05-10T21:15:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09fa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c377d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:33146/0 (socket says 172.21.15.50:33146) 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 -- 172.21.15.50:0/154603027 learned_addr learned my addr 172.21.15.50:0/154603027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09fa000640 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 0x564c377d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 -- 172.21.15.50:0/154603027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 msgr2=0x564c377d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c377d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 -- 172.21.15.50:0/154603027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 msgr2=0x564c377d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 0x564c377d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 -- 172.21.15.50:0/154603027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c377bd610 con 0x564c373f34d0 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09fa000640 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 0x564c377d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.336+0000 7f09faa00640 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003d00 tx=0x7f09ec02ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615940cacff60622 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.337+0000 7f09e7400640 1 -- 172.21.15.50:0/154603027 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec056070 con 0x564c373f34d0 2024-05-10T21:15:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.337+0000 7f09e7400640 1 -- 172.21.15.50:0/154603027 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ec0054e0 con 0x564c373f34d0 2024-05-10T21:15:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.337+0000 7f09e7400640 1 -- 172.21.15.50:0/154603027 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec035030 con 0x564c373f34d0 2024-05-10T21:15:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 -- 172.21.15.50:0/154603027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 msgr2=0x564c377c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003d00 tx=0x7f09ec02ab90 comp rx=0 tx=0).stop 2024-05-10T21:15:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 -- 172.21.15.50:0/154603027 shutdown_connections 2024-05-10T21:15:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c374141b0 0x564c377d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c373f34d0 0x564c377c7190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 --2- 172.21.15.50:0/154603027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c377d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 -- 172.21.15.50:0/154603027 >> 172.21.15.50:0/154603027 conn(0x564c37408300 msgr2=0x564c377ba230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 -- 172.21.15.50:0/154603027 shutdown_connections 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.338+0000 7f0a08c23740 1 -- 172.21.15.50:0/154603027 wait complete. 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 -- start start 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 Processor -- start 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c3783aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c374141b0 0x564c3783cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1ab7d0 con 0x564c373f34d0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1acc20 con 0x564c374141b0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1abe80 con 0x564c373eb2f0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.339+0000 7f0a08c23740 10 client.0 _ll_register_callbacks cb 0x7f09ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09fa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09fa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35612/0 (socket says 172.21.15.50:35612) 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09fa000640 1 -- 172.21.15.50:0/1989671134 learned_addr learned my addr 172.21.15.50:0/1989671134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c3783aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09fa000640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c374141b0 0x564c3783cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 -- 172.21.15.50:0/1989671134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 msgr2=0x564c378384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 -- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c374141b0 msgr2=0x564c3783cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c374141b0 0x564c3783cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09faa00640 1 -- 172.21.15.50:0/1989671134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3783d4a0 con 0x564c373f34d0 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.340+0000 7f09fa000640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.341+0000 7f09faa00640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c3783aa10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003d00 tx=0x7f09ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1507eee7a38bdca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.341+0000 7f09e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.341+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec060060 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.341+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09ec0054e0 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.341+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f09ec0038b0 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f09ec056070 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 10 client.16350 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c3783e690 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ec0054e0 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f09ec046a80 con 0x564c373f34d0 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 10 client.16350 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 20 client.16350 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373a81c0 0x564c37841040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 4 client.16350 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.342+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c37721fd0 con 0x564c373a81c0 2024-05-10T21:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.343+0000 7f09fa000640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373a81c0 0x564c37841040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.343+0000 7f09fa000640 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373a81c0 0x564c37841040 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851f6c958fa837cf server_cookie=e5485be5cdf4be29 in_seq=0 out_seq=0 2024-05-10T21:15:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.343+0000 7f09e4c00640 10 client.16350 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:32.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:32 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:32.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:32 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:32.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:32 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.547+0000 7f09e4c00640 1 -- 172.21.15.50:0/1989671134 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c37721fd0 con 0x564c373a81c0 2024-05-10T21:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.547+0000 7f09e4c00640 10 client.16350 handle_command_reply: tid=0 2024-05-10T21:15:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.547+0000 7f0a08c23740 1 client.16350 shutdown 2024-05-10T21:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 msgr2=0x564c3783aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c3783aa10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003d00 tx=0x7f09ec000bc0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 shutdown_connections 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x564c373a81c0 0x564c37841040 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c374141b0 0x564c3783cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c373f34d0 0x564c3783aa10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 --2- 172.21.15.50:0/1989671134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c373eb2f0 0x564c378384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.548+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 >> 172.21.15.50:0/1989671134 conn(0x564c37408300 msgr2=0x564c377b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.549+0000 7f09e4c00640 0 client.16350 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.549+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 shutdown_connections 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.549+0000 7f0a08c23740 1 -- 172.21.15.50:0/1989671134 wait complete. 2024-05-10T21:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.549+0000 7f0a08c23740 20 client.16350 trim_cache size 0 max 16384 2024-05-10T21:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:15:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.550+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 msgr2=0x7f09e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.550+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 0x7f09e00b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09f401a040 tx=0x7f09f4009f90 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.550+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 msgr2=0x7f09fc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.550+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09f400efd0 tx=0x7f09f400c6a0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 shutdown_connections 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f09e00b28b0 0x7f09e00b4dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc106bb0 0x7f09fc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09fc109720 0x7f09fc102610 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 --2- 172.21.15.50:0/2215553900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09fc1041c0 0x7f09fc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 >> 172.21.15.50:0/2215553900 conn(0x7f09fc0fdb60 msgr2=0x7f09fc104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 shutdown_connections 2024-05-10T21:15:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:32.551+0000 7f09fbe00640 1 -- 172.21.15.50:0/2215553900 wait complete. 2024-05-10T21:15:33.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:33 smithi092 ceph-mon[34415]: pgmap v661: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 645 op/s 2024-05-10T21:15:33.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:33 smithi142 ceph-mon[34846]: pgmap v661: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 645 op/s 2024-05-10T21:15:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:33 smithi050 ceph-mon[29546]: pgmap v661: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 645 op/s 2024-05-10T21:15:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:35 smithi092 ceph-mon[34415]: pgmap v662: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 585 op/s 2024-05-10T21:15:35.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:35 smithi142 ceph-mon[34846]: pgmap v662: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 585 op/s 2024-05-10T21:15:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:35 smithi050 ceph-mon[29546]: pgmap v662: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 585 op/s 2024-05-10T21:15:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:36 smithi092 ceph-mon[34415]: pgmap v663: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 530 op/s 2024-05-10T21:15:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:36 smithi142 ceph-mon[34846]: pgmap v663: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 530 op/s 2024-05-10T21:15:36.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:36 smithi050 ceph-mon[29546]: pgmap v663: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 530 op/s 2024-05-10T21:15:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:38 smithi092 ceph-mon[34415]: pgmap v664: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.3 MiB/s wr, 401 op/s 2024-05-10T21:15:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:38 smithi142 ceph-mon[34846]: pgmap v664: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.3 MiB/s wr, 401 op/s 2024-05-10T21:15:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:38 smithi050 ceph-mon[29546]: pgmap v664: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.3 MiB/s wr, 401 op/s 2024-05-10T21:15:40.126 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:39 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:15:40.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:39 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:15:40.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:39 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:15:41.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:40 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:15:41.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:40 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:15:41.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:40 smithi050 ceph-mon[29546]: pgmap v665: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 723 KiB/s wr, 282 op/s 2024-05-10T21:15:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:40 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:15:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:40 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:15:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:40 smithi092 ceph-mon[34415]: pgmap v665: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 723 KiB/s wr, 282 op/s 2024-05-10T21:15:41.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:40 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:15:41.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:40 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:15:41.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:40 smithi142 ceph-mon[34846]: pgmap v665: 97 pgs: 97 active+clean; 305 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 723 KiB/s wr, 282 op/s 2024-05-10T21:15:41.399 INFO:tasks.workunit.client.0.smithi050.stdout: 83 218541 297 138665 549 63899 209 2024-05-10T21:15:43.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:42 smithi092 ceph-mon[34415]: pgmap v666: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-05-10T21:15:43.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:42 smithi142 ceph-mon[34846]: pgmap v666: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-05-10T21:15:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:42 smithi050 ceph-mon[29546]: pgmap v666: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-05-10T21:15:45.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:44 smithi092 ceph-mon[34415]: pgmap v667: 97 pgs: 97 active+clean; 330 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-05-10T21:15:45.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:44 smithi142 ceph-mon[34846]: pgmap v667: 97 pgs: 97 active+clean; 330 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-05-10T21:15:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:44 smithi050 ceph-mon[29546]: pgmap v667: 97 pgs: 97 active+clean; 330 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-05-10T21:15:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.787+0000 7f1177e00640 1 -- 172.21.15.50:0/893685297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178106bb0 msgr2=0x7f1178109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.787+0000 7f1177e00640 1 --2- 172.21.15.50:0/893685297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178106bb0 0x7f1178109090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f116402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 -- 172.21.15.50:0/893685297 shutdown_connections 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 --2- 172.21.15.50:0/893685297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178106bb0 0x7f1178109090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 --2- 172.21.15.50:0/893685297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178109720 0x7f117810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 --2- 172.21.15.50:0/893685297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f11781065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 -- 172.21.15.50:0/893685297 >> 172.21.15.50:0/893685297 conn(0x7f11780fdb60 msgr2=0x7f1178100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 -- 172.21.15.50:0/893685297 shutdown_connections 2024-05-10T21:15:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.788+0000 7f1177e00640 1 -- 172.21.15.50:0/893685297 wait complete. 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 Processor -- start 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 -- start start 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f1178103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 0x7f11781020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178109720 0x7f1178102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f117810f560 con 0x7f1178106bb0 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f117810f3e0 con 0x7f11781041c0 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.789+0000 7f1177e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f117810f6e0 con 0x7f1178109720 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 0x7f11781020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f1178103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 0x7f11781020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43960/0 (socket says 172.21.15.50:43960) 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176000640 1 -- 172.21.15.50:0/578614382 learned_addr learned my addr 172.21.15.50:0/578614382 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1177400640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178109720 0x7f1178102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 -- 172.21.15.50:0/578614382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178109720 msgr2=0x7f1178102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178109720 0x7f1178102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 -- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 msgr2=0x7f11781020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 0x7f11781020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.790+0000 7f1176a00640 1 -- 172.21.15.50:0/578614382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178102d40 con 0x7f11781041c0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f1176a00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f1178103a30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9f0 tx=0x7f116800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f116f400640 1 -- 172.21.15.50:0/578614382 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800ccc0 con 0x7f11781041c0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11781a4250 con 0x7f11781041c0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f116f400640 1 -- 172.21.15.50:0/578614382 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168005020 con 0x7f11781041c0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f116f400640 1 -- 172.21.15.50:0/578614382 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168004a00 con 0x7f11781041c0 2024-05-10T21:15:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.791+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11781a4710 con 0x7f11781041c0 2024-05-10T21:15:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.795+0000 7f11849ec740 1 Processor -- start 2024-05-10T21:15:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 -- start start 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093777300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093777880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x56009377e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5600936db550 con 0x5600933d51b0 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560093680bc0 con 0x5600933b44d0 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.796+0000 7f11849ec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5600936d7510 con 0x5600933ac2f0 2024-05-10T21:15:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1177400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093777300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x56009377e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1177400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093777300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43964/0 (socket says 172.21.15.50:43964) 2024-05-10T21:15:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1177400640 1 -- 172.21.15.50:0/4178300459 learned_addr learned my addr 172.21.15.50:0/4178300459 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 -- 172.21.15.50:0/4178300459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 msgr2=0x560093777880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093777880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 -- 172.21.15.50:0/4178300459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 msgr2=0x560093777300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093777300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f1176000640 1 -- 172.21.15.50:0/4178300459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600933ed210 con 0x5600933b44d0 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.797+0000 7f116f400640 1 -- 172.21.15.50:0/578614382 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1168004ba0 con 0x7f11781041c0 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.798+0000 7f1176000640 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x56009377e780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f11781033d0 tx=0x7f1164031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e136eb27b438e911 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.798+0000 7f1163e00640 1 -- 172.21.15.50:0/4178300459 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035030 con 0x5600933b44d0 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.798+0000 7f1163e00640 1 -- 172.21.15.50:0/4178300459 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1164040070 con 0x5600933b44d0 2024-05-10T21:15:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.798+0000 7f1163e00640 1 -- 172.21.15.50:0/4178300459 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164003970 con 0x5600933b44d0 2024-05-10T21:15:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.799+0000 7f116f400640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 0x7f115c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f116f400640 1 -- 172.21.15.50:0/578614382 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f11680e21a0 con 0x7f11781041c0 2024-05-10T21:15:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 -- 172.21.15.50:0/4178300459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 msgr2=0x56009377e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x56009377e780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f11781033d0 tx=0x7f1164031b10 comp rx=0 tx=0).stop 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f1177400640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 0x7f115c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 -- 172.21.15.50:0/4178300459 shutdown_connections 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093777880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x56009377e780 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 --2- 172.21.15.50:0/4178300459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093777300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 -- 172.21.15.50:0/4178300459 >> 172.21.15.50:0/4178300459 conn(0x5600933c9300 msgr2=0x560093772c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 -- 172.21.15.50:0/4178300459 shutdown_connections 2024-05-10T21:15:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.800+0000 7f11849ec740 1 -- 172.21.15.50:0/4178300459 wait complete. 2024-05-10T21:15:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.801+0000 7f11849ec740 1 -- start start 2024-05-10T21:15:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.801+0000 7f1177400640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 0x7f115c0b4d70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5600937784d0 tx=0x7f1170009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.801+0000 7f11849ec740 1 Processor -- start 2024-05-10T21:15:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 0x5600937895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56009377f170 con 0x5600933d51b0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560093694660 con 0x5600933ac2f0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56009377f2f0 con 0x5600933b44d0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f11849ec740 10 client.0 _ll_register_callbacks cb 0x56009365e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f1176000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f1176000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44390/0 (socket says 172.21.15.50:44390) 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.802+0000 7f1176000640 1 -- 172.21.15.50:0/1348448082 learned_addr learned my addr 172.21.15.50:0/1348448082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176a00640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 0x5600937895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 -- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 msgr2=0x5600937895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 0x5600937895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 -- 172.21.15.50:0/1348448082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 msgr2=0x560093790a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1177400640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 -- 172.21.15.50:0/1348448082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600934b9620 con 0x5600933b44d0 2024-05-10T21:15:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176a00640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 0x5600937895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:15:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1177400640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1176000640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11640058c0 tx=0x7f1164031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e6cf778ee5f07d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.803+0000 7f1160c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035070 con 0x5600933b44d0 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164040070 con 0x5600933b44d0 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f11640036f0 con 0x5600933b44d0 2024-05-10T21:15:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f11640038f0 con 0x5600933b44d0 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f11849ec740 10 client.26449 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.804+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56009377f1c0 con 0x5600933b44d0 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1164040070 con 0x5600933b44d0 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1164045c00 con 0x5600933b44d0 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 10 client.26449 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 20 client.26449 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560093393d80 0x560093796c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 4 client.26449 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.805+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600936e9160 con 0x560093393d80 2024-05-10T21:15:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.806+0000 7f1176a00640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560093393d80 0x560093796c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.806+0000 7f1176a00640 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560093393d80 0x560093796c50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a0ca1769e52886 server_cookie=ba54163063ac07d6 in_seq=0 out_seq=0 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.806+0000 7f1160c00640 10 client.26449 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f1160c00640 1 -- 172.21.15.50:0/1348448082 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600936e9160 con 0x560093393d80 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f1160c00640 10 client.26449 handle_command_reply: tid=0 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 client.26449 shutdown 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 msgr2=0x560093789b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11640058c0 tx=0x7f1164031850 comp rx=0 tx=0).stop 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 shutdown_connections 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x560093393d80 0x560093796c50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x560093789b10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 --2- 172.21.15.50:0/1348448082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933ac2f0 0x5600937895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 >> 172.21.15.50:0/1348448082 conn(0x5600933c9300 msgr2=0x560093771c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.807+0000 7f1160c00640 0 client.26449 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.808+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 shutdown_connections 2024-05-10T21:15:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.808+0000 7f11849ec740 1 -- 172.21.15.50:0/1348448082 wait complete. 2024-05-10T21:15:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.808+0000 7f11849ec740 20 client.26449 trim_cache size 0 max 16384 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.833+0000 7f11849ec740 1 Processor -- start 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 -- start start 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x560093790990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11781ab750 con 0x5600933d51b0 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f11781a8fc0 con 0x5600933b44d0 2024-05-10T21:15:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f11849ec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11781a6e30 con 0x5600933ac2f0 2024-05-10T21:15:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f1176a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f1177400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f1176a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44394/0 (socket says 172.21.15.50:44394) 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.834+0000 7f1176a00640 1 -- 172.21.15.50:0/4094889177 learned_addr learned my addr 172.21.15.50:0/4094889177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176000640 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x560093790990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 -- 172.21.15.50:0/4094889177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 msgr2=0x560093790990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x560093790990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 -- 172.21.15.50:0/4094889177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 msgr2=0x560093790ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 -- 172.21.15.50:0/4094889177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009377e620 con 0x5600933ac2f0 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1176a00640 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11680a9720 tx=0x7f1168005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5fa264a1c22639 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.835+0000 7f1163400640 1 -- 172.21.15.50:0/4094889177 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680c2070 con 0x5600933ac2f0 2024-05-10T21:15:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.836+0000 7f1163400640 1 -- 172.21.15.50:0/4094889177 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680a9790 con 0x5600933ac2f0 2024-05-10T21:15:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.836+0000 7f1163400640 1 -- 172.21.15.50:0/4094889177 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801c040 con 0x5600933ac2f0 2024-05-10T21:15:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.836+0000 7f11849ec740 1 -- 172.21.15.50:0/4094889177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 msgr2=0x560093772d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.836+0000 7f11849ec740 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11680a9720 tx=0x7f1168005460 comp rx=0 tx=0).stop 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 -- 172.21.15.50:0/4094889177 shutdown_connections 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933d51b0 0x560093790ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933b44d0 0x560093790990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 --2- 172.21.15.50:0/4094889177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933ac2f0 0x560093772d60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 -- 172.21.15.50:0/4094889177 >> 172.21.15.50:0/4094889177 conn(0x5600933c9300 msgr2=0x560093772650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 -- 172.21.15.50:0/4094889177 shutdown_connections 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 -- 172.21.15.50:0/4094889177 wait complete. 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 -- start start 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.837+0000 7f11849ec740 1 Processor -- start 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x56009378ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933d51b0 0x56009378f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11781a7480 con 0x5600933ac2f0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f11781a88f0 con 0x5600933d51b0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11781a7b50 con 0x5600933b44d0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f11849ec740 10 client.0 _ll_register_callbacks cb 0x7f1170009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f1176a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f1176a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43984/0 (socket says 172.21.15.50:43984) 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f1176000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x56009378ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f1176a00640 1 -- 172.21.15.50:0/3457256625 learned_addr learned my addr 172.21.15.50:0/3457256625 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.838+0000 7f1177400640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933d51b0 0x56009378f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 -- 172.21.15.50:0/3457256625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 msgr2=0x56009378ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x56009378ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 -- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933d51b0 msgr2=0x56009378f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933d51b0 0x56009378f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 -- 172.21.15.50:0/3457256625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009378ff40 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176000640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x56009378ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1176a00640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f11680a9490 tx=0x7f1168005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeae3f1b88ddc04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1160c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.839+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680c2070 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f11680bebe0 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11680a9790 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f116801c040 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f11849ec740 10 client.16365 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560093791240 con 0x5600933ac2f0 2024-05-10T21:15:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680a9790 con 0x5600933ac2f0 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.840+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f11680b2160 con 0x5600933ac2f0 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 10 client.16365 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 20 client.16365 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600933691c0 0x560093801410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 4 client.16365 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:15:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600936e2fd0 con 0x5600933691c0 2024-05-10T21:15:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.841+0000 7f1176000640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600933691c0 0x560093801410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.842+0000 7f1176000640 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600933691c0 0x560093801410 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56fb6049c72b8b1 server_cookie=cc8408f72ec45eab in_seq=0 out_seq=0 2024-05-10T21:15:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:45.842+0000 7f1160c00640 10 client.16365 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.504+0000 7f1160c00640 1 -- 172.21.15.50:0/3457256625 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600936e2fd0 con 0x5600933691c0 2024-05-10T21:15:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.504+0000 7f1160c00640 10 client.16365 handle_command_reply: tid=0 2024-05-10T21:15:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.504+0000 7f11849ec740 1 client.16365 shutdown 2024-05-10T21:15:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.507+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 msgr2=0x56009378e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.507+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f11680a9490 tx=0x7f1168005720 comp rx=0 tx=0).stop 2024-05-10T21:15:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 shutdown_connections 2024-05-10T21:15:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5600933691c0 0x560093801410 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600933b44d0 0x56009378ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600933d51b0 0x56009378f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 --2- 172.21.15.50:0/3457256625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600933ac2f0 0x56009378e510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 >> 172.21.15.50:0/3457256625 conn(0x5600933c9300 msgr2=0x56009379b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.508+0000 7f1160c00640 0 client.16365 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.509+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 shutdown_connections 2024-05-10T21:15:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.509+0000 7f11849ec740 1 -- 172.21.15.50:0/3457256625 wait complete. 2024-05-10T21:15:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.509+0000 7f11849ec740 20 client.16365 trim_cache size 0 max 16384 2024-05-10T21:15:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:15:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:15:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:15:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:15:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 msgr2=0x7f115c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 0x7f115c0b4d70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5600937784d0 tx=0x7f1170009500 comp rx=0 tx=0).stop 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 msgr2=0x7f1178103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f1178103a30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9f0 tx=0x7f116800eeb0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 shutdown_connections 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f115c0b2860 0x7f115c0b4d70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178109720 0x7f1178102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1178106bb0 0x7f11781020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 --2- 172.21.15.50:0/578614382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11781041c0 0x7f1178103a30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.513+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 >> 172.21.15.50:0/578614382 conn(0x7f11780fdb60 msgr2=0x7f1178104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.514+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 shutdown_connections 2024-05-10T21:15:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:46.514+0000 7f1177e00640 1 -- 172.21.15.50:0/578614382 wait complete. 2024-05-10T21:15:47.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:46 smithi092 ceph-mon[34415]: pgmap v668: 97 pgs: 97 active+clean; 335 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-05-10T21:15:47.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:46 smithi142 ceph-mon[34846]: pgmap v668: 97 pgs: 97 active+clean; 335 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-05-10T21:15:47.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:46 smithi050 ceph-mon[29546]: pgmap v668: 97 pgs: 97 active+clean; 335 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-05-10T21:15:49.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:48 smithi092 ceph-mon[34415]: pgmap v669: 97 pgs: 97 active+clean; 337 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-10T21:15:49.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:48 smithi142 ceph-mon[34846]: pgmap v669: 97 pgs: 97 active+clean; 337 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-10T21:15:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:48 smithi050 ceph-mon[29546]: pgmap v669: 97 pgs: 97 active+clean; 337 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-05-10T21:15:51.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:50 smithi092 ceph-mon[34415]: pgmap v670: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-05-10T21:15:51.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:50 smithi142 ceph-mon[34846]: pgmap v670: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-05-10T21:15:51.399 INFO:tasks.workunit.client.0.smithi050.stdout: 98 33255 874 23212 761 9233 22 2024-05-10T21:15:51.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:50 smithi050 ceph-mon[29546]: pgmap v670: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-05-10T21:15:53.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:52 smithi092 ceph-mon[34415]: pgmap v671: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-05-10T21:15:53.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:52 smithi142 ceph-mon[34846]: pgmap v671: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-05-10T21:15:53.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:52 smithi050 ceph-mon[29546]: pgmap v671: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-05-10T21:15:55.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:54 smithi092 ceph-mon[34415]: pgmap v672: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-05-10T21:15:55.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:54 smithi142 ceph-mon[34846]: pgmap v672: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-05-10T21:15:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:54 smithi050 ceph-mon[29546]: pgmap v672: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-05-10T21:15:57.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:56 smithi092 ceph-mon[34415]: pgmap v673: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 146 op/s 2024-05-10T21:15:57.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:56 smithi142 ceph-mon[34846]: pgmap v673: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 146 op/s 2024-05-10T21:15:57.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:56 smithi050 ceph-mon[29546]: pgmap v673: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 146 op/s 2024-05-10T21:15:58.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:57 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:15:58.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:57 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:15:58.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:57 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:15:59.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:58 smithi092 ceph-mon[34415]: pgmap v674: 97 pgs: 97 active+clean; 364 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.7 MiB/s wr, 159 op/s 2024-05-10T21:15:59.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:58 smithi142 ceph-mon[34846]: pgmap v674: 97 pgs: 97 active+clean; 364 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.7 MiB/s wr, 159 op/s 2024-05-10T21:15:59.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:58 smithi050 ceph-mon[29546]: pgmap v674: 97 pgs: 97 active+clean; 364 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.7 MiB/s wr, 159 op/s 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 -- 172.21.15.50:0/118324987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c110070 msgr2=0x7fe61c112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 --2- 172.21.15.50:0/118324987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c110070 0x7fe61c112490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe614009be0 tx=0x7fe61402a6c0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 -- 172.21.15.50:0/118324987 shutdown_connections 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 --2- 172.21.15.50:0/118324987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c110070 0x7fe61c112490 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 --2- 172.21.15.50:0/118324987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c10b370 0x7fe61c10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 --2- 172.21.15.50:0/118324987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 -- 172.21.15.50:0/118324987 >> 172.21.15.50:0/118324987 conn(0x7fe61c078d30 msgr2=0x7fe61c07b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 -- 172.21.15.50:0/118324987 shutdown_connections 2024-05-10T21:15:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.816+0000 7fe622800640 1 -- 172.21.15.50:0/118324987 wait complete. 2024-05-10T21:15:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.817+0000 7fe622800640 1 Processor -- start 2024-05-10T21:15:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 -- start start 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 0x7fe61c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c110070 0x7fe61c1a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c115bb0 con 0x7fe61c110070 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c115a30 con 0x7fe61c10a400 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe622800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c115d30 con 0x7fe61c10b370 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe621400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.818+0000 7fe621e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c110070 0x7fe61c1a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 0x7fe61c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe621400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56346/0 (socket says 172.21.15.50:56346) 2024-05-10T21:15:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe621400640 1 -- 172.21.15.50:0/2989782968 learned_addr learned my addr 172.21.15.50:0/2989782968 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 -- 172.21.15.50:0/2989782968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 msgr2=0x7fe61c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 -- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c110070 msgr2=0x7fe61c1a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c110070 0x7fe61c1a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 -- 172.21.15.50:0/2989782968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c1a8230 con 0x7fe61c10b370 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe621400640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.819+0000 7fe620a00640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 0x7fe61c1a0c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e790 tx=0x7fe60800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.820+0000 7fe61a000640 1 -- 172.21.15.50:0/2989782968 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60800eea0 con 0x7fe61c10b370 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.820+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c1a8520 con 0x7fe61c10b370 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.820+0000 7fe61a000640 1 -- 172.21.15.50:0/2989782968 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe60800ccc0 con 0x7fe61c10b370 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.820+0000 7fe61a000640 1 -- 172.21.15.50:0/2989782968 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608018660 con 0x7fe61c10b370 2024-05-10T21:15:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.820+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a8ac0 con 0x7fe61c10b370 2024-05-10T21:15:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.825+0000 7fe61a000640 1 -- 172.21.15.50:0/2989782968 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe608010430 con 0x7fe61c10b370 2024-05-10T21:15:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.825+0000 7fe62ae50740 1 Processor -- start 2024-05-10T21:15:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.825+0000 7fe62ae50740 1 -- start start 2024-05-10T21:15:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b0c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b13780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561180a70550 con 0x5611807494d0 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561180a15bc0 con 0x5611807412f0 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe62ae50740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561180a6c510 con 0x56118076a1b0 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe621e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b0c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe621400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe621e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b0c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55376/0 (socket says 172.21.15.50:55376) 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe621e00640 1 -- 172.21.15.50:0/2858578474 learned_addr learned my addr 172.21.15.50:0/2858578474 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.826+0000 7fe620a00640 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b13780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe61a000640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 0x7fe6040b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 -- 172.21.15.50:0/2858578474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 msgr2=0x561180b0c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b0c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 -- 172.21.15.50:0/2858578474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 msgr2=0x561180b13780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b13780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 -- 172.21.15.50:0/2858578474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561180782210 con 0x5611807412f0 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe61a000640 1 -- 172.21.15.50:0/2989782968 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe6080e23b0 con 0x7fe61c10b370 2024-05-10T21:15:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621e00640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 0x7fe6040b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.827+0000 7fe621400640 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b0c880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe6100114c0 tx=0x7fe610011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=367d83f17acd7380 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.828+0000 7fe60e600640 1 -- 172.21.15.50:0/2858578474 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000f5c0 con 0x5611807412f0 2024-05-10T21:15:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.828+0000 7fe60e600640 1 -- 172.21.15.50:0/2858578474 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61000fc10 con 0x5611807412f0 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.828+0000 7fe60e600640 1 -- 172.21.15.50:0/2858578474 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610019d80 con 0x5611807412f0 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 -- 172.21.15.50:0/2858578474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 msgr2=0x561180b0c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe621e00640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 0x7fe6040b4cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe614009420 tx=0x7fe614009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b0c880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe6100114c0 tx=0x7fe610011980 comp rx=0 tx=0).stop 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 -- 172.21.15.50:0/2858578474 shutdown_connections 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b13780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b0c880 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 --2- 172.21.15.50:0/2858578474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b0c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 -- 172.21.15.50:0/2858578474 >> 172.21.15.50:0/2858578474 conn(0x56118075e300 msgr2=0x561180b07c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.829+0000 7fe62ae50740 1 -- 172.21.15.50:0/2858578474 shutdown_connections 2024-05-10T21:15:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.830+0000 7fe62ae50740 1 -- 172.21.15.50:0/2858578474 wait complete. 2024-05-10T21:15:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.830+0000 7fe62ae50740 1 -- start start 2024-05-10T21:15:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 Processor -- start 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807412f0 0x561180b15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807494d0 0x561180b16390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561180b141d0 con 0x5611807412f0 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561180a29660 con 0x5611807494d0 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561180b14350 con 0x56118076a1b0 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.831+0000 7fe62ae50740 10 client.0 _ll_register_callbacks cb 0x5611809f3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807412f0 0x561180b15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55390/0 (socket says 172.21.15.50:55390) 2024-05-10T21:15:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 -- 172.21.15.50:0/3842930016 learned_addr learned my addr 172.21.15.50:0/3842930016 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe620a00640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807494d0 0x561180b16390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 -- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807494d0 msgr2=0x561180b16390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807494d0 0x561180b16390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 -- 172.21.15.50:0/3842930016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807412f0 msgr2=0x561180b15e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807412f0 0x561180b15e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.832+0000 7fe621e00640 1 -- 172.21.15.50:0/3842930016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118084e620 con 0x56118076a1b0 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe621e00640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe614030730 tx=0x7fe614030990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf34b2bcbcb532f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe602a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614036040 con 0x56118076a1b0 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61000fc10 con 0x56118076a1b0 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe614004040 con 0x56118076a1b0 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe61405a050 con 0x56118076a1b0 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe62ae50740 10 client.26467 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.833+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561180b202e0 con 0x56118076a1b0 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.834+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61000fc10 con 0x56118076a1b0 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.834+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe61403f8d0 con 0x56118076a1b0 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 10 client.26467 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 10 client.26467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 20 client.26467 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561180728d80 0x561180b239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 4 client.26467 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561180a7e160 con 0x561180728d80 2024-05-10T21:15:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.835+0000 7fe621400640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561180728d80 0x561180b239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.836+0000 7fe621400640 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561180728d80 0x561180b239f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e16c68c5a31b7d1 server_cookie=58a16d48399711f8 in_seq=0 out_seq=0 2024-05-10T21:15:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.836+0000 7fe602a00640 10 client.26467 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe602a00640 1 -- 172.21.15.50:0/3842930016 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561180a7e160 con 0x561180728d80 2024-05-10T21:15:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe602a00640 10 client.26467 handle_command_reply: tid=0 2024-05-10T21:15:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 client.26467 shutdown 2024-05-10T21:15:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 msgr2=0x561180b1d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe614030730 tx=0x7fe614030990 comp rx=0 tx=0).stop 2024-05-10T21:15:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 shutdown_connections 2024-05-10T21:15:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561180728d80 0x561180b239f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56118076a1b0 0x561180b1d2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807494d0 0x561180b16390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3842930016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807412f0 0x561180b15e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 >> 172.21.15.50:0/3842930016 conn(0x56118075e300 msgr2=0x561180b0aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe602a00640 0 client.26467 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.837+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 shutdown_connections 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.838+0000 7fe62ae50740 1 -- 172.21.15.50:0/3842930016 wait complete. 2024-05-10T21:15:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.838+0000 7fe62ae50740 20 client.26467 trim_cache size 0 max 16384 2024-05-10T21:15:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.861+0000 7fe62ae50740 1 Processor -- start 2024-05-10T21:15:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 -- start start 2024-05-10T21:15:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807412f0 0x561180b10260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 0x561180b1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b3f60 con 0x5611807494d0 2024-05-10T21:15:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b17d0 con 0x56118076a1b0 2024-05-10T21:15:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe62ae50740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1af620 con 0x5611807412f0 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe620a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe621400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807412f0 0x561180b10260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe620a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39504/0 (socket says 172.21.15.50:39504) 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.862+0000 7fe620a00640 1 -- 172.21.15.50:0/1386905573 learned_addr learned my addr 172.21.15.50:0/1386905573 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe621e00640 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 0x561180b1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe620a00640 1 -- 172.21.15.50:0/1386905573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807412f0 msgr2=0x561180b10260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe620a00640 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807412f0 0x561180b10260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe620a00640 1 -- 172.21.15.50:0/1386905573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 msgr2=0x561180b1d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe620a00640 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 0x561180b1d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe620a00640 1 -- 172.21.15.50:0/1386905573 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561180b0be10 con 0x5611807494d0 2024-05-10T21:15:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.863+0000 7fe621e00640 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 0x561180b1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:15:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.864+0000 7fe620a00640 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6080a5ad0 tx=0x7fe6080a9460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ba4fe96cc151df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.864+0000 7fe60dc00640 1 -- 172.21.15.50:0/1386905573 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a5040 con 0x5611807494d0 2024-05-10T21:15:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.864+0000 7fe60dc00640 1 -- 172.21.15.50:0/1386905573 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6080a9560 con 0x5611807494d0 2024-05-10T21:15:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.864+0000 7fe60dc00640 1 -- 172.21.15.50:0/1386905573 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080e5c40 con 0x5611807494d0 2024-05-10T21:15:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 -- 172.21.15.50:0/1386905573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 msgr2=0x561180b107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:15:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6080a5ad0 tx=0x7fe6080a9460 comp rx=0 tx=0).stop 2024-05-10T21:15:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 -- 172.21.15.50:0/1386905573 shutdown_connections 2024-05-10T21:15:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56118076a1b0 0x561180b1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611807494d0 0x561180b107a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 --2- 172.21.15.50:0/1386905573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807412f0 0x561180b10260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 -- 172.21.15.50:0/1386905573 >> 172.21.15.50:0/1386905573 conn(0x56118075e300 msgr2=0x561180b0c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 -- 172.21.15.50:0/1386905573 shutdown_connections 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.866+0000 7fe62ae50740 1 -- 172.21.15.50:0/1386905573 wait complete. 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.867+0000 7fe62ae50740 1 -- start start 2024-05-10T21:15:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.867+0000 7fe62ae50740 1 Processor -- start 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56118076a1b0 0x561180b1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1afc90 con 0x56118076a1b0 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b1100 con 0x5611807412f0 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b0360 con 0x5611807494d0 2024-05-10T21:15:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe62ae50740 10 client.0 _ll_register_callbacks cb 0x7fe610005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:15:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe620a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe621e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56118076a1b0 0x561180b1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe620a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55406/0 (socket says 172.21.15.50:55406) 2024-05-10T21:15:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe621400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.868+0000 7fe620a00640 1 -- 172.21.15.50:0/3784462471 learned_addr learned my addr 172.21.15.50:0/3784462471 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 -- 172.21.15.50:0/3784462471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 msgr2=0x561180b1a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 -- 172.21.15.50:0/3784462471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56118076a1b0 msgr2=0x561180b1ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56118076a1b0 0x561180b1ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 -- 172.21.15.50:0/3784462471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561180b034a0 con 0x5611807412f0 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe620a00640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:15:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe621400640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b1da80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe6100095e0 tx=0x7fe61000ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0df333157b9e36d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.869+0000 7fe602a00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.870+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610031060 con 0x5611807412f0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.870+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fe610016040 con 0x5611807412f0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.870+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6080a9560 con 0x5611807412f0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.871+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe610011260 con 0x5611807412f0 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.871+0000 7fe62ae50740 10 client.26522 fetch_fsmap learned FSMap version 33 2024-05-10T21:15:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.871+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561180b8ae70 con 0x5611807412f0 2024-05-10T21:15:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.871+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6080a9560 con 0x5611807412f0 2024-05-10T21:15:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.871+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe610005500 con 0x5611807412f0 2024-05-10T21:15:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 10 client.26522 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 20 client.26522 populate_metadata read hostname 'smithi050' 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611806fe1c0 0x561180b8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 4 client.26522 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.872+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561180a77fd0 con 0x5611806fe1c0 2024-05-10T21:15:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.873+0000 7fe620a00640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611806fe1c0 0x561180b8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:15:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.873+0000 7fe620a00640 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611806fe1c0 0x561180b8d6b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784d7bf76cd73fb0 server_cookie=697d72952fe71285 in_seq=0 out_seq=0 2024-05-10T21:15:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:15:59.873+0000 7fe602a00640 10 client.26522 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:00.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:59 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:16:00.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:59 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:16:00.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:15:59 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:16:00.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:59 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:16:00.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:59 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:16:00.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:15:59 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:16:00.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:16:00.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:16:00.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:15:59 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:16:01.283 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:16:01.358 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:01 smithi050 ceph-mon[29546]: pgmap v675: 97 pgs: 97 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.2 MiB/s wr, 170 op/s 2024-05-10T21:16:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:01 smithi092 ceph-mon[34415]: pgmap v675: 97 pgs: 97 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.2 MiB/s wr, 170 op/s 2024-05-10T21:16:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:01 smithi142 ceph-mon[34846]: pgmap v675: 97 pgs: 97 active+clean; 373 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.2 MiB/s wr, 170 op/s 2024-05-10T21:16:01.399 INFO:tasks.workunit.client.0.smithi050.stdout: 111 41667 522 27494 570 9161 127 2024-05-10T21:16:01.801 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:03.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:03 smithi050 ceph-mon[29546]: pgmap v676: 97 pgs: 97 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-10T21:16:03.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:03 smithi092 ceph-mon[34415]: pgmap v676: 97 pgs: 97 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-10T21:16:03.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:03 smithi142 ceph-mon[34846]: pgmap v676: 97 pgs: 97 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-05-10T21:16:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 -- 172.21.15.50:0/738639098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 msgr2=0x7fa8c41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 --2- 172.21.15.50:0/738639098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c41100a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b0d0 tx=0x7fa8bc00b3e0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 -- 172.21.15.50:0/738639098 shutdown_connections 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 --2- 172.21.15.50:0/738639098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c41034c0 0x7fa8c41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 --2- 172.21.15.50:0/738639098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c41100a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 --2- 172.21.15.50:0/738639098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c4107df0 0x7fa8c4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.328+0000 7fa8ca200700 1 -- 172.21.15.50:0/738639098 >> 172.21.15.50:0/738639098 conn(0x7fa8c40fedc0 msgr2=0x7fa8c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.329+0000 7fa8ca200700 1 -- 172.21.15.50:0/738639098 shutdown_connections 2024-05-10T21:16:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.329+0000 7fa8ca200700 1 -- 172.21.15.50:0/738639098 wait complete. 2024-05-10T21:16:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.330+0000 7fa8ca200700 1 Processor -- start 2024-05-10T21:16:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.330+0000 7fa8ca200700 1 -- start start 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c4107df0 0x7fa8c40ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c40ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4113370 con 0x7fa8c4108740 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41131f0 con 0x7fa8c4107df0 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8ca200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41134f0 con 0x7fa8c41034c0 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34784/0 (socket says 172.21.15.50:34784) 2024-05-10T21:16:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8c2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c4107df0 0x7fa8c40ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.331+0000 7fa8c3400700 1 -- 172.21.15.50:0/3977380489 learned_addr learned my addr 172.21.15.50:0/3977380489 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c40ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 -- 172.21.15.50:0/3977380489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c4107df0 msgr2=0x7fa8c40ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c4107df0 0x7fa8c40ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 -- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 msgr2=0x7fa8c40ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c40ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 -- 172.21.15.50:0/3977380489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a9190 con 0x7fa8c41034c0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.332+0000 7fa8c3400700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400e9a0 tx=0x7fa8b400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.333+0000 7fa8bbe00700 1 -- 172.21.15.50:0/3977380489 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400cbd0 con 0x7fa8c41034c0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.333+0000 7fa8bbe00700 1 -- 172.21.15.50:0/3977380489 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b400cd70 con 0x7fa8c41034c0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.333+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a9480 con 0x7fa8c41034c0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.333+0000 7fa8bbe00700 1 -- 172.21.15.50:0/3977380489 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4018680 con 0x7fa8c41034c0 2024-05-10T21:16:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.333+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a9a10 con 0x7fa8c41034c0 2024-05-10T21:16:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.338+0000 7fa8d8d1db80 1 Processor -- start 2024-05-10T21:16:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.338+0000 7fa8d8d1db80 1 -- start start 2024-05-10T21:16:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8bbe00700 1 -- 172.21.15.50:0/3977380489 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa8b4010970 con 0x7fa8c41034c0 2024-05-10T21:16:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fa81c25a20 con 0x55fa81b85e40 2024-05-10T21:16:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa81bc3760 con 0x55fa81d039f0 2024-05-10T21:16:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fa81c219e0 con 0x55fa81d09830 2024-05-10T21:16:04.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8c2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8c2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36938/0 (socket says 172.21.15.50:36938) 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.339+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3239885560 learned_addr learned my addr 172.21.15.50:0/3239885560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3239885560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 msgr2=0x55fa81d034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3239885560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 msgr2=0x55fa81d092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3239885560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa81b728d0 con 0x55fa81d039f0 2024-05-10T21:16:04.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.340+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c60 tx=0x7fa8b0009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30acbfb078076fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8b8c00700 1 -- 172.21.15.50:0/3239885560 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0010be0 con 0x55fa81d039f0 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8bbe00700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 0x7fa8a80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8b8c00700 1 -- 172.21.15.50:0/3239885560 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00044d0 con 0x55fa81d039f0 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8b8c00700 1 -- 172.21.15.50:0/3239885560 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0011440 con 0x55fa81d039f0 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8bbe00700 1 -- 172.21.15.50:0/3977380489 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa8b40e2520 con 0x7fa8c41034c0 2024-05-10T21:16:04.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.341+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 0x7fa8a80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3239885560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 msgr2=0x55fa81d03e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c60 tx=0x7fa8b0009f70 comp rx=0 tx=0).stop 2024-05-10T21:16:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3239885560 shutdown_connections 2024-05-10T21:16:04.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d03e00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3239885560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3239885560 >> 172.21.15.50:0/3239885560 conn(0x55fa81bcaf70 msgr2=0x55fa81cfe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.342+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 0x7fa8a80b4e10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009fd0 tx=0x7fa8bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.343+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3239885560 shutdown_connections 2024-05-10T21:16:04.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.343+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3239885560 wait complete. 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.343+0000 7fa8d8d1db80 1 -- start start 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 Processor -- start 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d0cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fa81d10310 con 0x55fa81b85e40 2024-05-10T21:16:04.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa81b8bb20 con 0x55fa81d039f0 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fa81d10490 con 0x55fa81d09830 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.344+0000 7fa8d8d1db80 10 client.0 _ll_register_callbacks cb 0x55fa81bccc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d0cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d0cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41354/0 (socket says 172.21.15.50:41354) 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3400700 1 -- 172.21.15.50:0/4091393855 learned_addr learned my addr 172.21.15.50:0/4091393855 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:04.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 -- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 msgr2=0x55fa81d0d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d0d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 -- 172.21.15.50:0/4091393855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 msgr2=0x55fa81d0cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d0cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.345+0000 7fa8c3e00700 1 -- 172.21.15.50:0/4091393855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa81beb420 con 0x55fa81d09830 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.346+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d0d9e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc00b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f4a501dd2b890e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.346+0000 7fa8ad600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.346+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02d020 con 0x55fa81d09830 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.346+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b00044d0 con 0x55fa81d09830 2024-05-10T21:16:04.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.346+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa8bc007a20 con 0x55fa81d09830 2024-05-10T21:16:04.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.347+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc007c20 con 0x55fa81d09830 2024-05-10T21:16:04.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.347+0000 7fa8d8d1db80 10 client.26485 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:04.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.347+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa81d1adc0 con 0x55fa81d09830 2024-05-10T21:16:04.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.347+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00044d0 con 0x55fa81d09830 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa8bc011070 con 0x55fa81d09830 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 10 client.26485 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 20 client.26485 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d1d5d0 0x55fa81d1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 4 client.26485 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa81cd01c0 con 0x55fa81d1d5d0 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.348+0000 7fa8c3400700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d1d5d0 0x55fa81d1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.349+0000 7fa8c3400700 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d1d5d0 0x55fa81d1fa50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d31fefea5f0a2b9 server_cookie=10356cb286111560 in_seq=0 out_seq=0 2024-05-10T21:16:04.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.349+0000 7fa8ad600700 10 client.26485 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.518+0000 7fe602a00640 1 -- 172.21.15.50:0/3784462471 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561180a77fd0 con 0x5611806fe1c0 2024-05-10T21:16:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.518+0000 7fe602a00640 10 client.26522 handle_command_reply: tid=0 2024-05-10T21:16:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.519+0000 7fe62ae50740 1 client.26522 shutdown 2024-05-10T21:16:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.518+0000 7fa8ad600700 1 -- 172.21.15.50:0/4091393855 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa81cd01c0 con 0x55fa81d1d5d0 2024-05-10T21:16:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.518+0000 7fa8ad600700 10 client.26485 handle_command_reply: tid=0 2024-05-10T21:16:04.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.519+0000 7fa8d8d1db80 1 client.26485 shutdown 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.519+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 msgr2=0x561180b1da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.519+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b1da80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe6100095e0 tx=0x7fe61000ff50 comp rx=0 tx=0).stop 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 shutdown_connections 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5611806fe1c0 0x561180b8d6b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611807494d0 0x561180b1a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56118076a1b0 0x561180b1ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe602a00640 0 client.26522 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 --2- 172.21.15.50:0/3784462471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611807412f0 0x561180b1da80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 >> 172.21.15.50:0/3784462471 conn(0x56118075e300 msgr2=0x561180b0c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 shutdown_connections 2024-05-10T21:16:04.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 msgr2=0x55fa81d0d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d0d9e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc00b3e0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 1 -- 172.21.15.50:0/3784462471 wait complete. 2024-05-10T21:16:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fe62ae50740 20 client.26522 trim_cache size 0 max 16384 2024-05-10T21:16:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:16:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:16:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:16:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 shutdown_connections 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d1d5d0 0x55fa81d1fa50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81d09830 0x55fa81d0d9e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81b85e40 0x55fa81d0cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/4091393855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d0d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 >> 172.21.15.50:0/4091393855 conn(0x55fa81bcaf70 msgr2=0x55fa81d024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.520+0000 7fa8ad600700 0 client.26485 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:04.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.521+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 shutdown_connections 2024-05-10T21:16:04.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.521+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/4091393855 wait complete. 2024-05-10T21:16:04.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.521+0000 7fa8d8d1db80 20 client.26485 trim_cache size 0 max 16384 2024-05-10T21:16:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 msgr2=0x7fe6040b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 0x7fe6040b4cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe614009420 tx=0x7fe614009500 comp rx=0 tx=0).stop 2024-05-10T21:16:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 msgr2=0x7fe61c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 0x7fe61c1a0c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e790 tx=0x7fe60800ec50 comp rx=0 tx=0).stop 2024-05-10T21:16:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 shutdown_connections 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe6040b27e0 0x7fe6040b4cf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe61c10b370 0x7fe61c1a0c30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe61c110070 0x7fe61c1a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 --2- 172.21.15.50:0/2989782968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe61c10a400 0x7fe61c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 >> 172.21.15.50:0/2989782968 conn(0x7fe61c078d30 msgr2=0x7fe61c10c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 shutdown_connections 2024-05-10T21:16:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:04.522+0000 7fe622800640 1 -- 172.21.15.50:0/2989782968 wait complete. 2024-05-10T21:16:04.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.548+0000 7fa8d8d1db80 1 Processor -- start 2024-05-10T21:16:04.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 -- start start 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b3700 con 0x55fa81d09830 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b0f30 con 0x55fa81d039f0 2024-05-10T21:16:04.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.549+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41aeda0 con 0x55fa81b85e40 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:36952/0 (socket says 172.21.15.50:36952) 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3650666855 learned_addr learned my addr 172.21.15.50:0/3650666855 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34814/0 (socket says 172.21.15.50:34814) 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3650666855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 msgr2=0x55fa81d034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3650666855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 msgr2=0x55fa81d25250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c2a00700 1 -- 172.21.15.50:0/3650666855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa81d046b0 con 0x55fa81d039f0 2024-05-10T21:16:04.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.550+0000 7fa8c3400700 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:04.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.551+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c60 tx=0x7fa8b0009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95044ee1479e9615 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.551+0000 7fa8afe00700 1 -- 172.21.15.50:0/3650666855 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0010a60 con 0x55fa81d039f0 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.551+0000 7fa8afe00700 1 -- 172.21.15.50:0/3650666855 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00054e0 con 0x55fa81d039f0 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.551+0000 7fa8afe00700 1 -- 172.21.15.50:0/3650666855 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0004ca0 con 0x55fa81d039f0 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3650666855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 msgr2=0x55fa81d19360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009c60 tx=0x7fa8b0009740 comp rx=0 tx=0).stop 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3650666855 shutdown_connections 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/3650666855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d19360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.552+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3650666855 >> 172.21.15.50:0/3650666855 conn(0x55fa81bcaf70 msgr2=0x55fa81d0bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.553+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3650666855 shutdown_connections 2024-05-10T21:16:04.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.553+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/3650666855 wait complete. 2024-05-10T21:16:04.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.553+0000 7fa8d8d1db80 1 -- start start 2024-05-10T21:16:04.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.554+0000 7fa8d8d1db80 1 Processor -- start 2024-05-10T21:16:04.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d24f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41af3f0 con 0x55fa81d09830 2024-05-10T21:16:04.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b0860 con 0x55fa81d039f0 2024-05-10T21:16:04.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41afac0 con 0x55fa81b85e40 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8d8d1db80 10 client.0 _ll_register_callbacks cb 0x55fa81bccc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8c3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34816/0 (socket says 172.21.15.50:34816) 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8c3400700 1 -- 172.21.15.50:0/2478542028 learned_addr learned my addr 172.21.15.50:0/2478542028 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d24f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.555+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 -- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 msgr2=0x55fa81d24f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d24f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 -- 172.21.15.50:0/2478542028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 msgr2=0x55fa81d25490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 -- 172.21.15.50:0/2478542028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa81d10440 con 0x55fa81b85e40 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3e00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:16:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d24f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:04.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8c3400700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40a9b00 tx=0x7fa8b40dff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7e139c3c7ad7cff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:04.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8ad600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:04.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40a92e0 con 0x55fa81b85e40 2024-05-10T21:16:04.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fa8b40b2750 con 0x55fa81b85e40 2024-05-10T21:16:04.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.556+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b00054e0 con 0x55fa81b85e40 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8b40a5300 con 0x55fa81b85e40 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 10 client.26491 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa81d87e20 con 0x55fa81b85e40 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00054e0 con 0x55fa81b85e40 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa8b40e20b0 con 0x55fa81b85e40 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 10 client.26491 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 10 client.26491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:04.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 20 client.26491 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d88430 0x55fa81d8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.557+0000 7fa8d8d1db80 4 client.26491 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.558+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa81c33400 con 0x55fa81d88430 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.558+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d88430 0x55fa81d8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.558+0000 7fa8c2a00700 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d88430 0x55fa81d8a850 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670ede8c662fec12 server_cookie=3e55e229778f5fbc in_seq=0 out_seq=0 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.558+0000 7fa8ad600700 10 client.26491 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8ad600700 1 -- 172.21.15.50:0/2478542028 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa81c33400 con 0x55fa81d88430 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8ad600700 10 client.26491 handle_command_reply: tid=0 2024-05-10T21:16:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 client.26491 shutdown 2024-05-10T21:16:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 msgr2=0x55fa81d24a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40a9b00 tx=0x7fa8b40dff20 comp rx=0 tx=0).stop 2024-05-10T21:16:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 shutdown_connections 2024-05-10T21:16:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55fa81d88430 0x55fa81d8a850 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa81d09830 0x55fa81d25490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8ad600700 0 client.26491 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa81b85e40 0x55fa81d24a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 --2- 172.21.15.50:0/2478542028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa81d039f0 0x55fa81d24f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 >> 172.21.15.50:0/2478542028 conn(0x55fa81bcaf70 msgr2=0x55fa81d0bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.559+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 shutdown_connections 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.560+0000 7fa8d8d1db80 1 -- 172.21.15.50:0/2478542028 wait complete. 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.560+0000 7fa8d8d1db80 20 client.26491 trim_cache size 0 max 16384 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:16:04.558 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:16:04.559 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:16:04.559 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:16:04.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 msgr2=0x7fa8a80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 0x7fa8a80b4e10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009fd0 tx=0x7fa8bc00b040 comp rx=0 tx=0).stop 2024-05-10T21:16:04.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 msgr2=0x7fa8c40ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400e9a0 tx=0x7fa8b400ed60 comp rx=0 tx=0).stop 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 shutdown_connections 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa8a80b2900 0x7fa8a80b4e10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8c4108740 0x7fa8c40ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c41034c0 0x7fa8c40ff2a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 --2- 172.21.15.50:0/3977380489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c4107df0 0x7fa8c40ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 >> 172.21.15.50:0/3977380489 conn(0x7fa8c40fedc0 msgr2=0x7fa8c406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 shutdown_connections 2024-05-10T21:16:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:04.561+0000 7fa8ca200700 1 -- 172.21.15.50:0/3977380489 wait complete. 2024-05-10T21:16:05.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:05 smithi092 ceph-mon[34415]: pgmap v677: 97 pgs: 97 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-05-10T21:16:05.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:05 smithi142 ceph-mon[34846]: pgmap v677: 97 pgs: 97 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-05-10T21:16:05.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:05 smithi050 ceph-mon[29546]: pgmap v677: 97 pgs: 97 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-05-10T21:16:05.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:16:05.703 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:16:06.181 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:07.291 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:07 smithi050 ceph-mon[29546]: pgmap v678: 97 pgs: 97 active+clean; 420 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 368 op/s 2024-05-10T21:16:07.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:07 smithi092 ceph-mon[34415]: pgmap v678: 97 pgs: 97 active+clean; 420 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 368 op/s 2024-05-10T21:16:07.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:07 smithi142 ceph-mon[34846]: pgmap v678: 97 pgs: 97 active+clean; 420 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 368 op/s 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 -- 172.21.15.50:0/385181199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e941045f0 msgr2=0x7f2e94104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 --2- 172.21.15.50:0/385181199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e941045f0 0x7f2e94104a00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009920 tx=0x7f2e88009c30 comp rx=0 tx=0).stop 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 -- 172.21.15.50:0/385181199 shutdown_connections 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 --2- 172.21.15.50:0/385181199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94109870 0x7f2e941111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 --2- 172.21.15.50:0/385181199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e941045f0 0x7f2e94104a00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 --2- 172.21.15.50:0/385181199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e94108f20 0x7f2e94109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 -- 172.21.15.50:0/385181199 >> 172.21.15.50:0/385181199 conn(0x7f2e94100040 msgr2=0x7f2e94102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:08.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.690+0000 7f2e9be00700 1 -- 172.21.15.50:0/385181199 shutdown_connections 2024-05-10T21:16:08.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.691+0000 7f2e9be00700 1 -- 172.21.15.50:0/385181199 wait complete. 2024-05-10T21:16:08.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.692+0000 7f2e9be00700 1 Processor -- start 2024-05-10T21:16:08.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.692+0000 7f2e9be00700 1 -- start start 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 0x7f2e94075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 0x7f2e94076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941144f0 con 0x7f2e94108f20 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94114370 con 0x7f2e941045f0 2024-05-10T21:16:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e9be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94114670 con 0x7f2e94109870 2024-05-10T21:16:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e98a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e98a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41392/0 (socket says 172.21.15.50:41392) 2024-05-10T21:16:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.693+0000 7f2e98a00700 1 -- 172.21.15.50:0/993540970 learned_addr learned my addr 172.21.15.50:0/993540970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.694+0000 7f2e99e00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 0x7f2e94076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.694+0000 7f2e99400700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 0x7f2e94075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 -- 172.21.15.50:0/993540970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 msgr2=0x7f2e94076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 0x7f2e94076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 -- 172.21.15.50:0/993540970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 msgr2=0x7f2e94075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 0x7f2e94075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 -- 172.21.15.50:0/993540970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e941a9280 con 0x7f2e94108f20 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e99e00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 0x7f2e94076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e99400700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 0x7f2e94075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.695+0000 7f2e98a00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e9f0 tx=0x7f2e8000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.696+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000c9c0 con 0x7f2e94108f20 2024-05-10T21:16:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.696+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8000cb60 con 0x7f2e94108f20 2024-05-10T21:16:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.696+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80018680 con 0x7f2e94108f20 2024-05-10T21:16:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.696+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e941a9570 con 0x7f2e94108f20 2024-05-10T21:16:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.696+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941a9a60 con 0x7f2e94108f20 2024-05-10T21:16:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.698+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e940512c0 con 0x7f2e94108f20 2024-05-10T21:16:08.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.701+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2e80010640 con 0x7f2e94108f20 2024-05-10T21:16:08.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.704+0000 7f2e92000700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 0x7f2e7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:08.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.705+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f2e800e29d0 con 0x7f2e94108f20 2024-05-10T21:16:08.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.705+0000 7f2e99400700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 0x7f2e7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:08.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.706+0000 7f2e99400700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 0x7f2e7c0b4e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098f0 tx=0x7f2e88019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:08.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:08.707+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e800aa6c0 con 0x7f2e94108f20 2024-05-10T21:16:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.133+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e94076ff0 con 0x7f2e94108f20 2024-05-10T21:16:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.134+0000 7f2e92000700 1 -- 172.21.15.50:0/993540970 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11855 (secure 0 0 0) 0x7f2e800a9fc0 con 0x7f2e94108f20 2024-05-10T21:16:09.132 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:16:09.132 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:14:34.384991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24502,24520]},"id":1}]} 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.137+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 msgr2=0x7f2e7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.137+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 0x7f2e7c0b4e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098f0 tx=0x7f2e88019040 comp rx=0 tx=0).stop 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 msgr2=0x7f2e94076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e9f0 tx=0x7f2e8000ed00 comp rx=0 tx=0).stop 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 shutdown_connections 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f2e7c0b2900 0x7f2e7c0b4e10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e941045f0 0x7f2e94075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e94109870 0x7f2e94076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 --2- 172.21.15.50:0/993540970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e94108f20 0x7f2e94076000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 >> 172.21.15.50:0/993540970 conn(0x7f2e94100040 msgr2=0x7f2e94102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 shutdown_connections 2024-05-10T21:16:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:09.138+0000 7f2e9be00700 1 -- 172.21.15.50:0/993540970 wait complete. 2024-05-10T21:16:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 33 2024-05-10T21:16:09.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:09 smithi092 ceph-mon[34415]: pgmap v679: 97 pgs: 97 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 356 op/s 2024-05-10T21:16:09.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:09 smithi142 ceph-mon[34846]: pgmap v679: 97 pgs: 97 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 356 op/s 2024-05-10T21:16:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:09 smithi050 ceph-mon[29546]: pgmap v679: 97 pgs: 97 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 356 op/s 2024-05-10T21:16:10.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:16:10.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:10 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/993540970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:16:10.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:10 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/993540970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:16:10.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:10 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/993540970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:16:10.639 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:11.131 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:11 smithi050 ceph-mon[29546]: pgmap v680: 97 pgs: 97 active+clean; 429 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-05-10T21:16:11.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:11 smithi092 ceph-mon[34415]: pgmap v680: 97 pgs: 97 active+clean; 429 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-05-10T21:16:11.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:11 smithi142 ceph-mon[34846]: pgmap v680: 97 pgs: 97 active+clean; 429 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-05-10T21:16:11.399 INFO:tasks.workunit.client.0.smithi050.stdout: 134 8936 1373 6083 1215 1351 709 2024-05-10T21:16:12.098 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:12 smithi050 ceph-mon[29546]: pgmap v681: 97 pgs: 97 active+clean; 431 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2024-05-10T21:16:12.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:12 smithi092 ceph-mon[34415]: pgmap v681: 97 pgs: 97 active+clean; 431 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2024-05-10T21:16:12.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:12 smithi142 ceph-mon[34846]: pgmap v681: 97 pgs: 97 active+clean; 431 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2024-05-10T21:16:13.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 -- 172.21.15.50:0/1255583181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4109870 msgr2=0x7fb2f41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1255583181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4109870 0x7fb2f41111d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b0d0 tx=0x7fb2ec00b3e0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 -- 172.21.15.50:0/1255583181 shutdown_connections 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1255583181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1255583181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4109870 0x7fb2f41111d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1255583181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2f4108f20 0x7fb2f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.252+0000 7fb2faa00700 1 -- 172.21.15.50:0/1255583181 >> 172.21.15.50:0/1255583181 conn(0x7fb2f4100040 msgr2=0x7fb2f4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.253+0000 7fb2faa00700 1 -- 172.21.15.50:0/1255583181 shutdown_connections 2024-05-10T21:16:13.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.253+0000 7fb2faa00700 1 -- 172.21.15.50:0/1255583181 wait complete. 2024-05-10T21:16:13.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.254+0000 7fb2faa00700 1 Processor -- start 2024-05-10T21:16:13.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.254+0000 7fb2faa00700 1 -- start start 2024-05-10T21:16:13.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4108f20 0x7fb2f41a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2f4109870 0x7fb2f41ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41144f0 con 0x7fb2f4109870 2024-05-10T21:16:13.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4114370 con 0x7fb2f41045f0 2024-05-10T21:16:13.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2faa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4114670 con 0x7fb2f4108f20 2024-05-10T21:16:13.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37008/0 (socket says 172.21.15.50:37008) 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2f3e00700 1 -- 172.21.15.50:0/1525811142 learned_addr learned my addr 172.21.15.50:0/1525811142 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2f3400700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4108f20 0x7fb2f41a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.255+0000 7fb2f3e00700 1 -- 172.21.15.50:0/1525811142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4108f20 msgr2=0x7fb2f41a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4108f20 0x7fb2f41a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f3e00700 1 -- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2f4109870 msgr2=0x7fb2f41ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2f4109870 0x7fb2f41ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f3e00700 1 -- 172.21.15.50:0/1525811142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f41ad790 con 0x7fb2f41045f0 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9f0 tx=0x7fb2e400edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f0c00700 1 -- 172.21.15.50:0/1525811142 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400cbd0 con 0x7fb2f41045f0 2024-05-10T21:16:13.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f0c00700 1 -- 172.21.15.50:0/1525811142 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e400cd70 con 0x7fb2f41045f0 2024-05-10T21:16:13.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2f0c00700 1 -- 172.21.15.50:0/1525811142 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40056a0 con 0x7fb2f41045f0 2024-05-10T21:16:13.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.256+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41ad960 con 0x7fb2f41045f0 2024-05-10T21:16:13.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.257+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41addf0 con 0x7fb2f41045f0 2024-05-10T21:16:13.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.261+0000 7fb2f0c00700 1 -- 172.21.15.50:0/1525811142 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb2e4004750 con 0x7fb2f41045f0 2024-05-10T21:16:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.263+0000 7fb2f0c00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 0x7fb2d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.263+0000 7fb2f0c00700 1 -- 172.21.15.50:0/1525811142 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb2e40e24d0 con 0x7fb2f41045f0 2024-05-10T21:16:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.264+0000 7fb2f3400700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 0x7fb2d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.264+0000 7fb3094ffb80 1 Processor -- start 2024-05-10T21:16:13.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.264+0000 7fb3094ffb80 1 -- start start 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f6460fa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460fb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f64601ca20 con 0x55f6460fac50 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f645fba760 con 0x55f646100bf0 2024-05-10T21:16:13.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f6460189e0 con 0x55f645f7ce40 2024-05-10T21:16:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34852/0 (socket says 172.21.15.50:34852) 2024-05-10T21:16:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 -- 172.21.15.50:0/3527104027 learned_addr learned my addr 172.21.15.50:0/3527104027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3400700 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f6460fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 -- 172.21.15.50:0/3527104027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 msgr2=0x55f6460fa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f6460fa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 -- 172.21.15.50:0/3527104027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 msgr2=0x55f6460fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.265+0000 7fb2f3e00700 1 -- 172.21.15.50:0/3527104027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f645f698d0 con 0x55f645f7ce40 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2f3400700 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f6460fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:13.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2f3400700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 0x7fb2d80b4e10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4077a50 tx=0x7fb2e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40a9820 tx=0x7fb2e40a52b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16952c6add357e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2e9600700 1 -- 172.21.15.50:0/3527104027 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40c3070 con 0x55f645f7ce40 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2e9600700 1 -- 172.21.15.50:0/3527104027 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e40aae50 con 0x55f645f7ce40 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.266+0000 7fb2e9600700 1 -- 172.21.15.50:0/3527104027 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40a9d80 con 0x55f645f7ce40 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3527104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 msgr2=0x55f6461006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40a9820 tx=0x7fb2e40a52b0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3527104027 shutdown_connections 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6461006b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3527104027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f6460fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.267+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3527104027 >> 172.21.15.50:0/3527104027 conn(0x55f645fbba80 msgr2=0x55f6460f5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.268+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3527104027 shutdown_connections 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.268+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3527104027 wait complete. 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.268+0000 7fb3094ffb80 1 -- start start 2024-05-10T21:16:13.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.268+0000 7fb3094ffb80 1 Processor -- start 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.269+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.269+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6461049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 0x55f646104f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37024/0 (socket says 172.21.15.50:37024) 2024-05-10T21:16:13.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb2f3e00700 1 -- 172.21.15.50:0/484308095 learned_addr learned my addr 172.21.15.50:0/484308095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f6461076c0 con 0x55f6460fac50 2024-05-10T21:16:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb2f8a00700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 0x55f646104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb2f3400700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6461049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f645f82b20 con 0x55f645f7ce40 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f646107840 con 0x55f646100bf0 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.270+0000 7fb3094ffb80 10 client.0 _ll_register_callbacks cb 0x55f645fb0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 -- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 msgr2=0x55f646104f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 0x55f646104f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 -- 172.21.15.50:0/484308095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 msgr2=0x55f6461049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6461049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f645fe2420 con 0x55f645f7ce40 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40a95e0 tx=0x7fb2e40b2850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73089c9fa8a4a2d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2de000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:13.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f8a00700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 0x55f646104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:13.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2f3400700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6461049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:13.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40c8070 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb2e40a8d80 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.271+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e40aae50 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2e40c3070 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 10 client.26558 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f646112390 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e40aae50 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e4004a40 con 0x55f645f7ce40 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 10 client.26558 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 10 client.26558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 20 client.26558 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f646114860 0x55f646116c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 4 client.26558 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.272+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6460c7070 con 0x55f646114860 2024-05-10T21:16:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.273+0000 7fb2f3400700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f646114860 0x55f646116c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.273+0000 7fb2f3400700 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f646114860 0x55f646116c80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ce68833d2565a54 server_cookie=6363800ea54c1de2 in_seq=0 out_seq=0 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.273+0000 7fb2de000700 10 client.26558 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.274+0000 7fb2de000700 1 -- 172.21.15.50:0/484308095 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6460c7070 con 0x55f646114860 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.274+0000 7fb2de000700 10 client.26558 handle_command_reply: tid=0 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 client.26558 shutdown 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 msgr2=0x55f646104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40a95e0 tx=0x7fb2e40b2850 comp rx=0 tx=0).stop 2024-05-10T21:16:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 shutdown_connections 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f646114860 0x55f646116c80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f645f7ce40 0x55f646104490 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f646100bf0 0x55f646104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/484308095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6461049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 >> 172.21.15.50:0/484308095 conn(0x55f645fbba80 msgr2=0x55f6460f9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.275+0000 7fb2de000700 0 client.26558 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.276+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 shutdown_connections 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.276+0000 7fb3094ffb80 1 -- 172.21.15.50:0/484308095 wait complete. 2024-05-10T21:16:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.276+0000 7fb3094ffb80 20 client.26558 trim_cache size 0 max 16384 2024-05-10T21:16:13.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.300+0000 7fb3094ffb80 1 Processor -- start 2024-05-10T21:16:13.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 -- start start 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6460fa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b7ae0 con 0x55f6460fac50 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b5310 con 0x55f646100bf0 2024-05-10T21:16:13.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b3160 con 0x55f645f7ce40 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.301+0000 7fb2f8a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37030/0 (socket says 172.21.15.50:37030) 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 -- 172.21.15.50:0/3024800381 learned_addr learned my addr 172.21.15.50:0/3024800381 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6460fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 -- 172.21.15.50:0/3024800381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 msgr2=0x55f6460fa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6460fa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 -- 172.21.15.50:0/3024800381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 msgr2=0x55f6460efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f8a00700 1 -- 172.21.15.50:0/3024800381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6460f00e0 con 0x55f646100bf0 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.302+0000 7fb2f3400700 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.303+0000 7fb2f8a00700 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009d40 tx=0x7fb2ec009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d04eef409b4a002 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.303+0000 7fb2e8c00700 1 -- 172.21.15.50:0/3024800381 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00baa0 con 0x55f646100bf0 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.303+0000 7fb2e8c00700 1 -- 172.21.15.50:0/3024800381 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec00bc40 con 0x55f646100bf0 2024-05-10T21:16:13.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.303+0000 7fb2e8c00700 1 -- 172.21.15.50:0/3024800381 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec007c30 con 0x55f646100bf0 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3024800381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 msgr2=0x55f646104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009d40 tx=0x7fb2ec009d70 comp rx=0 tx=0).stop 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3024800381 shutdown_connections 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f646100bf0 0x55f646104d50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f6460fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/3024800381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6460fac50 0x55f6460efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.304+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3024800381 >> 172.21.15.50:0/3024800381 conn(0x55f645fbba80 msgr2=0x55f6460fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.305+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3024800381 shutdown_connections 2024-05-10T21:16:13.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.305+0000 7fb3094ffb80 1 -- 172.21.15.50:0/3024800381 wait complete. 2024-05-10T21:16:13.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.306+0000 7fb3094ffb80 1 -- start start 2024-05-10T21:16:13.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.306+0000 7fb3094ffb80 1 Processor -- start 2024-05-10T21:16:13.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.306+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f64611a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.306+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 0x55f6461175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb3094ffb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f646100bf0 0x55f646117b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b37d0 con 0x55f646100bf0 2024-05-10T21:16:13.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b4c40 con 0x55f6460fac50 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb3094ffb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b3ea0 con 0x55f645f7ce40 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb3094ffb80 10 client.0 _ll_register_callbacks cb 0x55f645fb0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f64611a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 0x55f6461175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f64611a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34880/0 (socket says 172.21.15.50:34880) 2024-05-10T21:16:13.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3e00700 1 -- 172.21.15.50:0/417359800 learned_addr learned my addr 172.21.15.50:0/417359800 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 -- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 msgr2=0x55f64611a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f64611a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 -- 172.21.15.50:0/417359800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f646100bf0 msgr2=0x55f646117b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f646100bf0 0x55f646117b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.307+0000 7fb2f3400700 1 -- 172.21.15.50:0/417359800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6461182b0 con 0x55f6460fac50 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb2f3400700 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 0x55f6461175d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f6461077b0 tx=0x7fb2e00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ce95100ffff53f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb2de000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0029380 con 0x55f6460fac50 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb2e0029520 con 0x55f6460fac50 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ec00bc40 con 0x55f6460fac50 2024-05-10T21:16:13.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2e0039070 con 0x55f6460fac50 2024-05-10T21:16:13.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb3094ffb80 10 client.26564 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:13.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.308+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f64617ed40 con 0x55f6460fac50 2024-05-10T21:16:13.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec00bc40 con 0x55f6460fac50 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e0011b10 con 0x55f6460fac50 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 10 client.26564 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 10 client.26564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 20 client.26564 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f64617f3d0 0x55f6461817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 4 client.26564 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:16:13.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f64602a400 con 0x55f64617f3d0 2024-05-10T21:16:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.309+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f64617f3d0 0x55f6461817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.311+0000 7fb2f3e00700 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f64617f3d0 0x55f6461817f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71185dd629115e1c server_cookie=10a338ecd5500e3a in_seq=0 out_seq=0 2024-05-10T21:16:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.311+0000 7fb2de000700 10 client.26564 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:16:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.312+0000 7fb2de000700 1 -- 172.21.15.50:0/417359800 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f64602a400 con 0x55f64617f3d0 2024-05-10T21:16:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.312+0000 7fb2de000700 10 client.26564 handle_command_reply: tid=0 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.312+0000 7fb3094ffb80 1 client.26564 shutdown 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.312+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 msgr2=0x55f6461175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.312+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 0x55f6461175d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f6461077b0 tx=0x7fb2e00047c0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 shutdown_connections 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x55f64617f3d0 0x55f6461817f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f646100bf0 0x55f646117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f645f7ce40 0x55f64611a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 --2- 172.21.15.50:0/417359800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6460fac50 0x55f6461175d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 >> 172.21.15.50:0/417359800 conn(0x55f645fbba80 msgr2=0x55f6460fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb2de000700 0 client.26564 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 shutdown_connections 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.313+0000 7fb3094ffb80 1 -- 172.21.15.50:0/417359800 wait complete. 2024-05-10T21:16:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.314+0000 7fb3094ffb80 20 client.26564 trim_cache size 0 max 16384 2024-05-10T21:16:13.311 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:16:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.314+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 msgr2=0x7fb2d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.314+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 0x7fb2d80b4e10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4077a50 tx=0x7fb2e0009260 comp rx=0 tx=0).stop 2024-05-10T21:16:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.314+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 msgr2=0x7fb2f41a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.314+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9f0 tx=0x7fb2e400edb0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 shutdown_connections 2024-05-10T21:16:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb2d80b2900 0x7fb2d80b4e10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2f41045f0 0x7fb2f41a21d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2f4109870 0x7fb2f41ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 --2- 172.21.15.50:0/1525811142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2f4108f20 0x7fb2f41a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:13.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 >> 172.21.15.50:0/1525811142 conn(0x7fb2f4100040 msgr2=0x7fb2f410a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:13.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 shutdown_connections 2024-05-10T21:16:13.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:13.315+0000 7fb2faa00700 1 -- 172.21.15.50:0/1525811142 wait complete. 2024-05-10T21:16:14.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:16:15.097 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:15.256 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:14 smithi050 ceph-mon[29546]: pgmap v682: 97 pgs: 97 active+clean; 436 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 241 op/s 2024-05-10T21:16:15.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:14 smithi092 ceph-mon[34415]: pgmap v682: 97 pgs: 97 active+clean; 436 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 241 op/s 2024-05-10T21:16:15.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:14 smithi142 ceph-mon[34846]: pgmap v682: 97 pgs: 97 active+clean; 436 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 241 op/s 2024-05-10T21:16:17.203 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:16 smithi050 ceph-mon[29546]: pgmap v683: 97 pgs: 97 active+clean; 447 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2024-05-10T21:16:17.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:16 smithi092 ceph-mon[34415]: pgmap v683: 97 pgs: 97 active+clean; 447 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2024-05-10T21:16:17.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:16 smithi142 ceph-mon[34846]: pgmap v683: 97 pgs: 97 active+clean; 447 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 175 op/s 2024-05-10T21:16:17.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 -- 172.21.15.50:0/419245900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc0764b0 msgr2=0x7f8bcc114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 --2- 172.21.15.50:0/419245900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc114e30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009990 tx=0x7f8bb8009ca0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 -- 172.21.15.50:0/419245900 shutdown_connections 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 --2- 172.21.15.50:0/419245900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc115370 0x7f8bcc117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 --2- 172.21.15.50:0/419245900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 --2- 172.21.15.50:0/419245900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc114e30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.674+0000 7f8bd1600700 1 -- 172.21.15.50:0/419245900 >> 172.21.15.50:0/419245900 conn(0x7f8bcc100040 msgr2=0x7f8bcc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.675+0000 7f8bd1600700 1 -- 172.21.15.50:0/419245900 shutdown_connections 2024-05-10T21:16:17.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.675+0000 7f8bd1600700 1 -- 172.21.15.50:0/419245900 wait complete. 2024-05-10T21:16:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.676+0000 7f8bd1600700 1 Processor -- start 2024-05-10T21:16:17.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.676+0000 7f8bd1600700 1 -- start start 2024-05-10T21:16:17.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc115370 0x7f8bcc1a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11ac00 con 0x7f8bcc0764b0 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11aa80 con 0x7f8bcc115370 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bd1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc11ad80 con 0x7f8bcc075b60 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bca000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bcb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc115370 0x7f8bcc1a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bca000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49934/0 (socket says 172.21.15.50:49934) 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.677+0000 7f8bca000700 1 -- 172.21.15.50:0/3800986298 learned_addr learned my addr 172.21.15.50:0/3800986298 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bca000700 1 -- 172.21.15.50:0/3800986298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 msgr2=0x7f8bcc1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bca000700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bca000700 1 -- 172.21.15.50:0/3800986298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc115370 msgr2=0x7f8bcc1a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bca000700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc115370 0x7f8bcc1a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bca000700 1 -- 172.21.15.50:0/3800986298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc1a6520 con 0x7f8bcc0764b0 2024-05-10T21:16:17.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.678+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.679+0000 7f8bca000700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80057f0 tx=0x7f8bb80037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.679+0000 7f8bc3400700 1 -- 172.21.15.50:0/3800986298 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8027070 con 0x7f8bcc0764b0 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.679+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1a67b0 con 0x7f8bcc0764b0 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.679+0000 7f8bc3400700 1 -- 172.21.15.50:0/3800986298 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8003fc0 con 0x7f8bcc0764b0 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.679+0000 7f8bc3400700 1 -- 172.21.15.50:0/3800986298 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8022cf0 con 0x7f8bcc0764b0 2024-05-10T21:16:17.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.680+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc1ae130 con 0x7f8bcc0764b0 2024-05-10T21:16:17.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.684+0000 7f8bc3400700 1 -- 172.21.15.50:0/3800986298 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8bb802a070 con 0x7f8bcc0764b0 2024-05-10T21:16:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.685+0000 7f8bc3400700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 0x7f8bb00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.686+0000 7f8bc3400700 1 -- 172.21.15.50:0/3800986298 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8bb80e3590 con 0x7f8bcc0764b0 2024-05-10T21:16:17.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.686+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 0x7f8bb00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.687+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 0x7f8bb00b4e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009a00 tx=0x7f8bbc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.687+0000 7f8be01deb80 1 Processor -- start 2024-05-10T21:16:17.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 -- start start 2024-05-10T21:16:17.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd0d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd07710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd08060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4acc29a20 con 0x55a4acb89e40 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4acbc7760 con 0x55a4acd0dbf0 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.688+0000 7f8be01deb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4acc259e0 con 0x55a4acd07c50 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.689+0000 7f8bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd07710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.689+0000 7f8bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd07710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39092/0 (socket says 172.21.15.50:39092) 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.689+0000 7f8bca000700 1 -- 172.21.15.50:0/2547510005 learned_addr learned my addr 172.21.15.50:0/2547510005 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.689+0000 7f8bcb400700 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd08060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.689+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd0d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 -- 172.21.15.50:0/2547510005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 msgr2=0x55a4acd07710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd07710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 -- 172.21.15.50:0/2547510005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 msgr2=0x55a4acd0d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd0d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 -- 172.21.15.50:0/2547510005 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4acb768d0 con 0x55a4acd07c50 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bcb400700 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd08060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009f90 tx=0x7f8bc4009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612e0a7da6aeecb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bb7e00700 1 -- 172.21.15.50:0/2547510005 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4014620 con 0x55a4acd07c50 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bb7e00700 1 -- 172.21.15.50:0/2547510005 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc4019070 con 0x55a4acd07c50 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.690+0000 7f8bb7e00700 1 -- 172.21.15.50:0/2547510005 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400aaa0 con 0x55a4acd07c50 2024-05-10T21:16:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.691+0000 7f8be01deb80 1 -- 172.21.15.50:0/2547510005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 msgr2=0x55a4acd08060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.691+0000 7f8be01deb80 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd08060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009f90 tx=0x7f8bc4009630 comp rx=0 tx=0).stop 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 -- 172.21.15.50:0/2547510005 shutdown_connections 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd08060 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd0d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 --2- 172.21.15.50:0/2547510005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd07710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 -- 172.21.15.50:0/2547510005 >> 172.21.15.50:0/2547510005 conn(0x55a4acbc8a80 msgr2=0x55a4acd02790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 -- 172.21.15.50:0/2547510005 shutdown_connections 2024-05-10T21:16:17.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.692+0000 7f8be01deb80 1 -- 172.21.15.50:0/2547510005 wait complete. 2024-05-10T21:16:17.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.693+0000 7f8be01deb80 1 -- start start 2024-05-10T21:16:17.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.694+0000 7f8be01deb80 1 Processor -- start 2024-05-10T21:16:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.695+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.695+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd11880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8bcaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8bcaa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49960/0 (socket says 172.21.15.50:49960) 2024-05-10T21:16:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8bcaa00700 1 -- 172.21.15.50:0/239297767 learned_addr learned my addr 172.21.15.50:0/239297767 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd11dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4acd146c0 con 0x55a4acb89e40 2024-05-10T21:16:17.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a4acb8fb20 con 0x55a4acd0dbf0 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4acd14840 con 0x55a4acd07c50 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8be01deb80 10 client.0 _ll_register_callbacks cb 0x55a4acbbd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.696+0000 7f8bca000700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd11880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 -- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 msgr2=0x55a4acd11dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bcb400700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd11dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd11dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 -- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 msgr2=0x55a4acd11340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4acbef420 con 0x55a4acd07c50 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bcb400700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd11dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bca000700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd11880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005f50 tx=0x7f8bb8022770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b11c3b40a49641a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80c3020 con 0x55a4acd07c50 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.697+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8bb80b3760 con 0x55a4acd07c50 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.698+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc4019070 con 0x55a4acd07c50 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.698+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8bb80a63a0 con 0x55a4acd07c50 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.698+0000 7f8be01deb80 10 client.26521 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.698+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a4acd146c0 con 0x55a4acd07c50 2024-05-10T21:16:17.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc4019070 con 0x55a4acd07c50 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8bb80e30d0 con 0x55a4acd07c50 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 10 client.26521 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 10 client.26521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 20 client.26521 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd217b0 0x55a4acd23c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 4 client.26521 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.699+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4accd4070 con 0x55a4acd217b0 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.700+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd217b0 0x55a4acd23c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.700+0000 7f8bb4c00700 10 client.26521 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:16:17.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.700+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd217b0 0x55a4acd23c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82f4291ccb9fc99b server_cookie=a7db1fb7a694cae9 in_seq=0 out_seq=0 2024-05-10T21:16:17.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.702+0000 7f8bb4c00700 1 -- 172.21.15.50:0/239297767 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4accd4070 con 0x55a4acd217b0 2024-05-10T21:16:17.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.702+0000 7f8bb4c00700 10 client.26521 handle_command_reply: tid=0 2024-05-10T21:16:17.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.702+0000 7f8be01deb80 1 client.26521 shutdown 2024-05-10T21:16:17.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.702+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 msgr2=0x55a4acd11880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.702+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd11880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005f50 tx=0x7f8bb8022770 comp rx=0 tx=0).stop 2024-05-10T21:16:17.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 shutdown_connections 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd11dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd217b0 0x55a4acd23c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd11340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 --2- 172.21.15.50:0/239297767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd11880 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 >> 172.21.15.50:0/239297767 conn(0x55a4acbc8a80 msgr2=0x55a4acd06700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8bb4c00700 0 client.26521 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 shutdown_connections 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 1 -- 172.21.15.50:0/239297767 wait complete. 2024-05-10T21:16:17.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.703+0000 7f8be01deb80 20 client.26521 trim_cache size 0 max 16384 2024-05-10T21:16:17.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.721+0000 7f8be01deb80 1 Processor -- start 2024-05-10T21:16:17.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.721+0000 7f8be01deb80 1 -- start start 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd07710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd1d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b7ae0 con 0x55a4acb89e40 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b5310 con 0x55a4acd0dbf0 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8be01deb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b31a0 con 0x55a4acd07c50 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8bca000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8bca000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd0cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56238/0 (socket says 172.21.15.50:56238) 2024-05-10T21:16:17.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8bca000700 1 -- 172.21.15.50:0/963330707 learned_addr learned my addr 172.21.15.50:0/963330707 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8bcb400700 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd1d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.722+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd07710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 -- 172.21.15.50:0/963330707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 msgr2=0x55a4acd0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd0cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 -- 172.21.15.50:0/963330707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 msgr2=0x55a4acd1d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd1d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 -- 172.21.15.50:0/963330707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4acd041d0 con 0x55a4acb89e40 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.723+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd07710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a4acd243a0 tx=0x7f8bbc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd31cd3ecc6ff42e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.724+0000 7f8bb7400700 1 -- 172.21.15.50:0/963330707 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc024890 con 0x55a4acb89e40 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.724+0000 7f8bb7400700 1 -- 172.21.15.50:0/963330707 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bbc027150 con 0x55a4acb89e40 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.724+0000 7f8bb7400700 1 -- 172.21.15.50:0/963330707 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc022440 con 0x55a4acb89e40 2024-05-10T21:16:17.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 -- 172.21.15.50:0/963330707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 msgr2=0x55a4acd07710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd07710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a4acd243a0 tx=0x7f8bbc013000 comp rx=0 tx=0).stop 2024-05-10T21:16:17.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 -- 172.21.15.50:0/963330707 shutdown_connections 2024-05-10T21:16:17.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd0dbf0 0x55a4acd1d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acb89e40 0x55a4acd07710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 --2- 172.21.15.50:0/963330707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acd07c50 0x55a4acd0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 -- 172.21.15.50:0/963330707 >> 172.21.15.50:0/963330707 conn(0x55a4acbc8a80 msgr2=0x55a4acd0c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.725+0000 7f8be01deb80 1 -- 172.21.15.50:0/963330707 shutdown_connections 2024-05-10T21:16:17.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.726+0000 7f8be01deb80 1 -- 172.21.15.50:0/963330707 wait complete. 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.726+0000 7f8be01deb80 1 -- start start 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.726+0000 7f8be01deb80 1 Processor -- start 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 0x55a4acd19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 0x55a4acd1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 0x55a4acd28450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b37f0 con 0x55a4acd0dbf0 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b4c40 con 0x55a4acd07c50 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc1b3ea0 con 0x55a4acb89e40 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8be01deb80 10 client.0 _ll_register_callbacks cb 0x55a4acbbd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 0x55a4acd1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 0x55a4acd1a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39132/0 (socket says 172.21.15.50:39132) 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.727+0000 7f8bca000700 1 -- 172.21.15.50:0/3719212986 learned_addr learned my addr 172.21.15.50:0/3719212986 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.728+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 0x55a4acd19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.728+0000 7f8bcb400700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 0x55a4acd28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.728+0000 7f8bcaa00700 1 -- 172.21.15.50:0/3719212986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 msgr2=0x55a4acd1a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.728+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 0x55a4acd1a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.728+0000 7f8bcaa00700 1 -- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 msgr2=0x55a4acd28450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.729+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 0x55a4acd28450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.729+0000 7f8bcaa00700 1 -- 172.21.15.50:0/3719212986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4acd28990 con 0x55a4acb89e40 2024-05-10T21:16:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.729+0000 7f8bcb400700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 0x55a4acd28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.729+0000 7f8bcaa00700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 0x55a4acd19ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00db70 tx=0x7f8bbc027ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643a27133acfb38b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.729+0000 7f8bb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.730+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc03d060 con 0x55a4acb89e40 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.730+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f8bbc027510 con 0x55a4acb89e40 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.731+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bbc027150 con 0x55a4acb89e40 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.732+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc024890 con 0x55a4acb89e40 2024-05-10T21:16:17.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.732+0000 7f8be01deb80 10 client.26527 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:17.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.732+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a4acd8b800 con 0x55a4acb89e40 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.732+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bbc027150 con 0x55a4acb89e40 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8bbc013070 con 0x55a4acb89e40 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 10 client.26527 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 10 client.26527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 20 client.26527 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd8bea0 0x55a4acd8e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 4 client.26527 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:16:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.733+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4acc37400 con 0x55a4acd8bea0 2024-05-10T21:16:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.736+0000 7f8bca000700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd8bea0 0x55a4acd8e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.737+0000 7f8bca000700 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd8bea0 0x55a4acd8e2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99e9fa7238cd6605 server_cookie=1b46411a559fbeca in_seq=0 out_seq=0 2024-05-10T21:16:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.737+0000 7f8bb4c00700 10 client.26527 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:16:17.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8bb4c00700 1 -- 172.21.15.50:0/3719212986 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4acc37400 con 0x55a4acd8bea0 2024-05-10T21:16:17.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8bb4c00700 10 client.26527 handle_command_reply: tid=0 2024-05-10T21:16:17.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 client.26527 shutdown 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 msgr2=0x55a4acd19ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 0x55a4acd19ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00db70 tx=0x7f8bbc027ad0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 shutdown_connections 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x55a4acd8bea0 0x55a4acd8e2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4acd0dbf0 0x55a4acd28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4acb89e40 0x55a4acd19ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 --2- 172.21.15.50:0/3719212986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4acd07c50 0x55a4acd1a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 >> 172.21.15.50:0/3719212986 conn(0x55a4acbc8a80 msgr2=0x55a4acd0b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.741+0000 7f8bb4c00700 0 client.26527 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:16:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.742+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 shutdown_connections 2024-05-10T21:16:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.742+0000 7f8be01deb80 1 -- 172.21.15.50:0/3719212986 wait complete. 2024-05-10T21:16:17.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.742+0000 7f8be01deb80 20 client.26527 trim_cache size 0 max 16384 2024-05-10T21:16:17.739 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:16:17.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.743+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 msgr2=0x7f8bb00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.743+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 0x7f8bb00b4e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009a00 tx=0x7f8bbc008040 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.743+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 msgr2=0x7f8bcc1a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.743+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80057f0 tx=0x7f8bb80037b0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 shutdown_connections 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f8bb00b2900 0x7f8bb00b4e10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bcc115370 0x7f8bcc1a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bcc075b60 0x7f8bcc1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 --2- 172.21.15.50:0/3800986298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bcc0764b0 0x7f8bcc1a1bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 >> 172.21.15.50:0/3800986298 conn(0x7f8bcc100040 msgr2=0x7f8bcc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 shutdown_connections 2024-05-10T21:16:17.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:17.744+0000 7f8bd1600700 1 -- 172.21.15.50:0/3800986298 wait complete. 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 -- 172.21.15.50:0/892533787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10103c60 msgr2=0x7feb10104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 --2- 172.21.15.50:0/892533787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10103c60 0x7feb10104960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb00009860 tx=0x7feb0002a340 comp rx=0 tx=0).stop 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 -- 172.21.15.50:0/892533787 shutdown_connections 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 --2- 172.21.15.50:0/892533787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10109ba0 0x7feb1010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 --2- 172.21.15.50:0/892533787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 0x7feb101052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 --2- 172.21.15.50:0/892533787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10103c60 0x7feb10104960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 -- 172.21.15.50:0/892533787 >> 172.21.15.50:0/892533787 conn(0x7feb100fee50 msgr2=0x7feb10101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 -- 172.21.15.50:0/892533787 shutdown_connections 2024-05-10T21:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.804+0000 7feb14c00640 1 -- 172.21.15.50:0/892533787 wait complete. 2024-05-10T21:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.805+0000 7feb14c00640 1 Processor -- start 2024-05-10T21:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.805+0000 7feb14c00640 1 -- start start 2024-05-10T21:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 0x7feb1019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f740 con 0x7feb10104ea0 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f5c0 con 0x7feb10103c60 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb14c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010f8c0 con 0x7feb10109ba0 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56262/0 (socket says 172.21.15.50:56262) 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0de00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39160/0 (socket says 172.21.15.50:39160) 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0e800640 1 -- 172.21.15.50:0/2581503709 learned_addr learned my addr 172.21.15.50:0/2581503709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.806+0000 7feb0d400640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 0x7feb1019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 -- 172.21.15.50:0/2581503709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 msgr2=0x7feb101a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 -- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 msgr2=0x7feb1019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 0x7feb1019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 -- 172.21.15.50:0/2581503709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10074f80 con 0x7feb10103c60 2024-05-10T21:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0e800640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb0de00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb00009780 tx=0x7feb00032e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb06a00640 1 -- 172.21.15.50:0/2581503709 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00037040 con 0x7feb10103c60 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.807+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb10075210 con 0x7feb10103c60 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.808+0000 7feb06a00640 1 -- 172.21.15.50:0/2581503709 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb00030c50 con 0x7feb10103c60 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.808+0000 7feb06a00640 1 -- 172.21.15.50:0/2581503709 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00003850 con 0x7feb10103c60 2024-05-10T21:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.808+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb100756c0 con 0x7feb10103c60 2024-05-10T21:16:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.812+0000 7feb1bc38740 1 Processor -- start 2024-05-10T21:16:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 -- start start 2024-05-10T21:16:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7d11b0 0x55e69bb73100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7a82f0 0x55e69bb73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e69bad7550 con 0x55e69b7d11b0 2024-05-10T21:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e69ba7cbc0 con 0x55e69b7b04d0 2024-05-10T21:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.813+0000 7feb1bc38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e69bad3510 con 0x55e69b7a82f0 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39162/0 (socket says 172.21.15.50:39162) 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 -- 172.21.15.50:0/1746799323 learned_addr learned my addr 172.21.15.50:0/1746799323 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0e800640 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7d11b0 0x55e69bb73100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 -- 172.21.15.50:0/1746799323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7a82f0 msgr2=0x55e69bb73680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7a82f0 0x55e69bb73680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 -- 172.21.15.50:0/1746799323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7d11b0 msgr2=0x55e69bb73100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7d11b0 0x55e69bb73100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 -- 172.21.15.50:0/1746799323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e69b7e9210 con 0x55e69b7b04d0 2024-05-10T21:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.814+0000 7feb0d400640 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feafc009c90 tx=0x7feafc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852bb10bda3ce75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.815+0000 7feb06a00640 1 -- 172.21.15.50:0/2581503709 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb00043020 con 0x7feb10103c60 2024-05-10T21:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.815+0000 7feafb000640 1 -- 172.21.15.50:0/1746799323 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc004230 con 0x55e69b7b04d0 2024-05-10T21:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.815+0000 7feafb000640 1 -- 172.21.15.50:0/1746799323 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc0043d0 con 0x55e69b7b04d0 2024-05-10T21:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.815+0000 7feafb000640 1 -- 172.21.15.50:0/1746799323 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc019770 con 0x55e69b7b04d0 2024-05-10T21:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.816+0000 7feb1bc38740 1 -- 172.21.15.50:0/1746799323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 msgr2=0x55e69bb7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.816+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feafc009c90 tx=0x7feafc011810 comp rx=0 tx=0).stop 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.816+0000 7feb1bc38740 1 -- 172.21.15.50:0/1746799323 shutdown_connections 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7a82f0 0x55e69bb73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bb7a580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1746799323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7d11b0 0x55e69bb73100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 -- 172.21.15.50:0/1746799323 >> 172.21.15.50:0/1746799323 conn(0x55e69b7c5300 msgr2=0x55e69bb6ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 -- 172.21.15.50:0/1746799323 shutdown_connections 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb1bc38740 1 -- 172.21.15.50:0/1746799323 wait complete. 2024-05-10T21:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.817+0000 7feb06a00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 0x7feaf40b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.818+0000 7feb06a00640 1 -- 172.21.15.50:0/2581503709 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7feb00101540 con 0x7feb10103c60 2024-05-10T21:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.818+0000 7feb1bc38740 1 -- start start 2024-05-10T21:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.819+0000 7feb1bc38740 1 Processor -- start 2024-05-10T21:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.819+0000 7feb0d400640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 0x7feaf40b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 0x55e69bb859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb0d400640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 0x7feaf40b4c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feafc003db0 tx=0x7feafc009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7d11b0 0x55e69bb8c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb0de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb0de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49998/0 (socket says 172.21.15.50:49998) 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb0de00640 1 -- 172.21.15.50:0/2144616653 learned_addr learned my addr 172.21.15.50:0/2144616653 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e69bb7afd0 con 0x55e69b7a82f0 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e69ba90660 con 0x55e69b7d11b0 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e69bb7b150 con 0x55e69b7b04d0 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb0d400640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 0x55e69bb859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.820+0000 7feb1bc38740 10 client.0 _ll_register_callbacks cb 0x55e69ba5a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0e800640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7d11b0 0x55e69bb8c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0de00640 1 -- 172.21.15.50:0/2144616653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 msgr2=0x55e69bb859e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0de00640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 0x55e69bb859e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0de00640 1 -- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7d11b0 msgr2=0x55e69bb8c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0de00640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7d11b0 0x55e69bb8c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0de00640 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e69b8b5620 con 0x55e69b7a82f0 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.821+0000 7feb0d400640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 0x55e69bb859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.822+0000 7feb0de00640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb000c8b20 tx=0x7feb000c84e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d486da5e70bce54e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.822+0000 7feaef400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.822+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000328f0 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.822+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7feb00101320 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.823+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feafc0043d0 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.823+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb00030e00 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.823+0000 7feb1bc38740 10 client.16440 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.823+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e69bb8f850 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.824+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc0043d0 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.824+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7feb000d1560 con 0x55e69b7a82f0 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 10 client.16440 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 20 client.16440 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b78fd80 0x55e69bb92fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 4 client.16440 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e69bae5160 con 0x55e69b78fd80 2024-05-10T21:16:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.825+0000 7feb0d400640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b78fd80 0x55e69bb92fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.826+0000 7feb0d400640 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b78fd80 0x55e69bb92fd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17a1c4789c82a87 server_cookie=963ca09873b6db3b in_seq=0 out_seq=0 2024-05-10T21:16:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.826+0000 7feaef400640 10 client.16440 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.827+0000 7feaef400640 1 -- 172.21.15.50:0/2144616653 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e69bae5160 con 0x55e69b78fd80 2024-05-10T21:16:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.827+0000 7feaef400640 10 client.16440 handle_command_reply: tid=0 2024-05-10T21:16:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.827+0000 7feb1bc38740 1 client.16440 shutdown 2024-05-10T21:16:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.827+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 msgr2=0x55e69bb854a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.827+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb000c8b20 tx=0x7feb000c84e0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 shutdown_connections 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b78fd80 0x55e69bb92fd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7b04d0 0x55e69bb859e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7d11b0 0x55e69bb8c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 --2- 172.21.15.50:0/2144616653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb854a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 >> 172.21.15.50:0/2144616653 conn(0x55e69b7c5300 msgr2=0x55e69bb6da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feaef400640 0 client.16440 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 shutdown_connections 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 1 -- 172.21.15.50:0/2144616653 wait complete. 2024-05-10T21:16:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.828+0000 7feb1bc38740 20 client.16440 trim_cache size 0 max 16384 2024-05-10T21:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.848+0000 7feb1bc38740 1 Processor -- start 2024-05-10T21:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.848+0000 7feb1bc38740 1 -- start start 2024-05-10T21:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.848+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 0x55e69bb85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.848+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7b04d0 0x55e69bb8c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bb97790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb1bc38740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb101aba80 con 0x55e69b7b04d0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb1bc38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb101a92f0 con 0x55e69b7a82f0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb1bc38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb101a7140 con 0x55e69b7d11b0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bb97790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bb97790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56276/0 (socket says 172.21.15.50:56276) 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0e800640 1 -- 172.21.15.50:0/1687195263 learned_addr learned my addr 172.21.15.50:0/1687195263 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 0x55e69bb85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0d400640 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7b04d0 0x55e69bb8c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 -- 172.21.15.50:0/1687195263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 msgr2=0x55e69bb97790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bb97790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 -- 172.21.15.50:0/1687195263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7b04d0 msgr2=0x55e69bb8c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7b04d0 0x55e69bb8c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.849+0000 7feb0de00640 1 -- 172.21.15.50:0/1687195263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e69bb6f620 con 0x55e69b7a82f0 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.850+0000 7feb0de00640 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 0x55e69bb85780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feb000c8890 tx=0x7feb00005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c292cd875e504b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.850+0000 7feafa600640 1 -- 172.21.15.50:0/1687195263 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000c8300 con 0x55e69b7a82f0 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.850+0000 7feafa600640 1 -- 172.21.15.50:0/1687195263 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb00030e00 con 0x55e69b7a82f0 2024-05-10T21:16:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.850+0000 7feafa600640 1 -- 172.21.15.50:0/1687195263 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000c8b90 con 0x55e69b7a82f0 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 -- 172.21.15.50:0/1687195263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 msgr2=0x55e69bb85780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 0x55e69bb85780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feb000c8890 tx=0x7feb00005c90 comp rx=0 tx=0).stop 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 -- 172.21.15.50:0/1687195263 shutdown_connections 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bb97790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7b04d0 0x55e69bb8c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 --2- 172.21.15.50:0/1687195263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7a82f0 0x55e69bb85780 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 -- 172.21.15.50:0/1687195263 >> 172.21.15.50:0/1687195263 conn(0x55e69b7c5300 msgr2=0x55e69bb85070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 -- 172.21.15.50:0/1687195263 shutdown_connections 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.851+0000 7feb1bc38740 1 -- 172.21.15.50:0/1687195263 wait complete. 2024-05-10T21:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.852+0000 7feb1bc38740 1 -- start start 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.852+0000 7feb1bc38740 1 Processor -- start 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.852+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.852+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb1bc38740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bbf9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb1bc38740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb101a77b0 con 0x55e69b7a82f0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb1bc38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb101a8c20 con 0x55e69b7b04d0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb1bc38740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb101a7e80 con 0x55e69b7d11b0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb1bc38740 10 client.0 _ll_register_callbacks cb 0x7feb00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50014/0 (socket says 172.21.15.50:50014) 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39198/0 (socket says 172.21.15.50:39198) 2024-05-10T21:16:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0de00640 1 -- 172.21.15.50:0/3852238169 learned_addr learned my addr 172.21.15.50:0/3852238169 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0e800640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bbf9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 -- 172.21.15.50:0/3852238169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 msgr2=0x55e69bbf9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bbf9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 -- 172.21.15.50:0/3852238169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 msgr2=0x55e69bb8c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0d400640 1 -- 172.21.15.50:0/3852238169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e69bbfa060 con 0x55e69b7b04d0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.853+0000 7feb0de00640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.854+0000 7feb0d400640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feafc01b040 tx=0x7feafc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59282eb6d58687f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.854+0000 7feaef400640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.854+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc03e070 con 0x55e69b7b04d0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.854+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7feafc009700 con 0x55e69b7b04d0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.854+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb00030e00 con 0x55e69b7b04d0 2024-05-10T21:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feafc004390 con 0x55e69b7b04d0 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feb1bc38740 10 client.26594 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e69bbfb3c0 con 0x55e69b7b04d0 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb00030e00 con 0x55e69b7b04d0 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7feafc0374d0 con 0x55e69b7b04d0 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feb1bc38740 10 client.26594 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.855+0000 7feb1bc38740 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb1bc38740 20 client.26594 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b7651c0 0x55e69bbfdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb1bc38740 4 client.26594 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e69badefd0 con 0x55e69b7651c0 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb0de00640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b7651c0 0x55e69bbfdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feb0de00640 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b7651c0 0x55e69bbfdc20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4712c428a5cebcb5 server_cookie=1656023ef8033bc6 in_seq=0 out_seq=0 2024-05-10T21:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:17.856+0000 7feaef400640 10 client.26594 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:18 smithi050 ceph-mon[29546]: pgmap v684: 97 pgs: 97 active+clean; 453 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 123 op/s 2024-05-10T21:16:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feaef400640 1 -- 172.21.15.50:0/3852238169 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e69badefd0 con 0x55e69b7651c0 2024-05-10T21:16:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feaef400640 10 client.26594 handle_command_reply: tid=0 2024-05-10T21:16:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feb1bc38740 1 client.26594 shutdown 2024-05-10T21:16:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 msgr2=0x55e69bbf7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feafc01b040 tx=0x7feafc009260 comp rx=0 tx=0).stop 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 shutdown_connections 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.342+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e69b7651c0 0x55e69bbfdc20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feaef400640 0 client.26594 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e69b7d11b0 0x55e69bbf9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e69b7b04d0 0x55e69bbf7330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 --2- 172.21.15.50:0/3852238169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e69b7a82f0 0x55e69bb8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 >> 172.21.15.50:0/3852238169 conn(0x55e69b7c5300 msgr2=0x55e69bb71890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 shutdown_connections 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.343+0000 7feb1bc38740 1 -- 172.21.15.50:0/3852238169 wait complete. 2024-05-10T21:16:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.344+0000 7feb1bc38740 20 client.26594 trim_cache size 0 max 16384 2024-05-10T21:16:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:16:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:16:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:16:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:16:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 msgr2=0x7feaf40b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 0x7feaf40b4c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feafc003db0 tx=0x7feafc009f00 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 msgr2=0x7feb1019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb00009780 tx=0x7feb00032e30 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 shutdown_connections 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feaf40b2770 0x7feaf40b4c80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb10109ba0 0x7feb101a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb10104ea0 0x7feb1019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 --2- 172.21.15.50:0/2581503709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb10103c60 0x7feb1019a350 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 >> 172.21.15.50:0/2581503709 conn(0x7feb100fee50 msgr2=0x7feb10105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 shutdown_connections 2024-05-10T21:16:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:19.345+0000 7feb14c00640 1 -- 172.21.15.50:0/2581503709 wait complete. 2024-05-10T21:16:19.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:18 smithi092 ceph-mon[34415]: pgmap v684: 97 pgs: 97 active+clean; 453 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 123 op/s 2024-05-10T21:16:19.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:18 smithi142 ceph-mon[34846]: pgmap v684: 97 pgs: 97 active+clean; 453 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 123 op/s 2024-05-10T21:16:21.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:20 smithi092 ceph-mon[34415]: pgmap v685: 97 pgs: 97 active+clean; 463 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-05-10T21:16:21.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:20 smithi142 ceph-mon[34846]: pgmap v685: 97 pgs: 97 active+clean; 463 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-05-10T21:16:21.400 INFO:tasks.workunit.client.0.smithi050.stdout: 177 469 2040 418 1973 0 757 2024-05-10T21:16:21.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:20 smithi050 ceph-mon[29546]: pgmap v685: 97 pgs: 97 active+clean; 463 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-05-10T21:16:23.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:22 smithi092 ceph-mon[34415]: pgmap v686: 97 pgs: 97 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-05-10T21:16:23.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:22 smithi142 ceph-mon[34846]: pgmap v686: 97 pgs: 97 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-05-10T21:16:23.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:22 smithi050 ceph-mon[29546]: pgmap v686: 97 pgs: 97 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-05-10T21:16:25.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:24 smithi092 ceph-mon[34415]: pgmap v687: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 1.01k op/s 2024-05-10T21:16:25.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:24 smithi142 ceph-mon[34846]: pgmap v687: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 1.01k op/s 2024-05-10T21:16:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:24 smithi050 ceph-mon[29546]: pgmap v687: 97 pgs: 97 active+clean; 475 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 1.01k op/s 2024-05-10T21:16:26.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:16:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:26 smithi050 ceph-mon[29546]: pgmap v688: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-10T21:16:27.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:26 smithi092 ceph-mon[34415]: pgmap v688: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-10T21:16:27.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:26 smithi142 ceph-mon[34846]: pgmap v688: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-10T21:16:27.435 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:28 smithi050 ceph-mon[29546]: pgmap v689: 97 pgs: 97 active+clean; 605 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-10T21:16:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:28 smithi092 ceph-mon[34415]: pgmap v689: 97 pgs: 97 active+clean; 605 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-10T21:16:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:28 smithi142 ceph-mon[34846]: pgmap v689: 97 pgs: 97 active+clean; 605 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-10T21:16:29.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 -- 172.21.15.50:0/4194106852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c115180 msgr2=0x7f079c1175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 --2- 172.21.15.50:0/4194106852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c115180 0x7f079c1175a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f079400b0a0 tx=0x7f079400b3b0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 -- 172.21.15.50:0/4194106852 shutdown_connections 2024-05-10T21:16:29.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 --2- 172.21.15.50:0/4194106852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c075b60 0x7f079c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 --2- 172.21.15.50:0/4194106852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c115180 0x7f079c1175a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 --2- 172.21.15.50:0/4194106852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0764b0 0x7f079c114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 -- 172.21.15.50:0/4194106852 >> 172.21.15.50:0/4194106852 conn(0x7f079c079d10 msgr2=0x7f079c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.784+0000 7f07a1000700 1 -- 172.21.15.50:0/4194106852 shutdown_connections 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.785+0000 7f07a1000700 1 -- 172.21.15.50:0/4194106852 wait complete. 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.786+0000 7f07a1000700 1 Processor -- start 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 -- start start 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c0764b0 0x7f079c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 0x7f079c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f079c11a880 con 0x7f079c075b60 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f079c11a700 con 0x7f079c0764b0 2024-05-10T21:16:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.787+0000 7f07a1000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f079c11aa00 con 0x7f079c115180 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60640/0 (socket says 172.21.15.50:60640) 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079b400700 1 -- 172.21.15.50:0/2584357521 learned_addr learned my addr 172.21.15.50:0/2584357521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 0x7f079c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079aa00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c0764b0 0x7f079c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 -- 172.21.15.50:0/2584357521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c0764b0 msgr2=0x7f079c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c0764b0 0x7f079c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 -- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 msgr2=0x7f079c1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079be00700 1 -- 172.21.15.50:0/2584357521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c1a60c0 con 0x7f079c115180 2024-05-10T21:16:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.788+0000 7f079b400700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f079be00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 0x7f079c1a5a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f079400b070 tx=0x7f0794004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f0793e00700 1 -- 172.21.15.50:0/2584357521 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794021070 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f0793e00700 1 -- 172.21.15.50:0/2584357521 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0794007e30 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c1a6350 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f0793e00700 1 -- 172.21.15.50:0/2584357521 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940045d0 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.789+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c1adcd0 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.793+0000 7f0793e00700 1 -- 172.21.15.50:0/2584357521 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f079401d450 con 0x7f079c115180 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.794+0000 7f07b10dcb80 1 Processor -- start 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.794+0000 7f07b10dcb80 1 -- start start 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f0793e00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 0x7f07800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 0x55e89e1b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e89e0cba20 con 0x55e89e1aa690 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e89e069760 con 0x55e89e02be40 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e89e0c79e0 con 0x55e89e1aa280 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f0793e00700 1 -- 172.21.15.50:0/2584357521 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f079400f020 con 0x7f079c115180 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f079aa00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 0x7f07800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f079b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f079b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39782/0 (socket says 172.21.15.50:39782) 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.795+0000 7f079b400700 1 -- 172.21.15.50:0/698689337 learned_addr learned my addr 172.21.15.50:0/698689337 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079be00700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 0x55e89e1b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079aa00700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 -- 172.21.15.50:0/698689337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 msgr2=0x55e89e1b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 -- 172.21.15.50:0/698689337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 msgr2=0x55e89e1b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 0x55e89e1b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 -- 172.21.15.50:0/698689337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89e0188d0 con 0x55e89e02be40 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079be00700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 0x55e89e1b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.796+0000 7f079b400700 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f078c0117e0 tx=0x7f078c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f7798dc8bede4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.797+0000 7f079aa00700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 0x7f07800b4d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f079c116c90 tx=0x7f0788011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.797+0000 7f0790c00700 1 -- 172.21.15.50:0/698689337 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c01e070 con 0x55e89e02be40 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.797+0000 7f0790c00700 1 -- 172.21.15.50:0/698689337 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078c00f930 con 0x55e89e02be40 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.797+0000 7f0790c00700 1 -- 172.21.15.50:0/698689337 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0198f0 con 0x55e89e02be40 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 -- 172.21.15.50:0/698689337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 msgr2=0x55e89e1a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f078c0117e0 tx=0x7f078c011af0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 -- 172.21.15.50:0/698689337 shutdown_connections 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e02be40 0x55e89e1a9d40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e1aa690 0x55e89e1b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/698689337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 -- 172.21.15.50:0/698689337 >> 172.21.15.50:0/698689337 conn(0x55e89e06aa80 msgr2=0x55e89e1a4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.798+0000 7f07b10dcb80 1 -- 172.21.15.50:0/698689337 shutdown_connections 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.799+0000 7f07b10dcb80 1 -- 172.21.15.50:0/698689337 wait complete. 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.799+0000 7f07b10dcb80 1 -- start start 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.799+0000 7f07b10dcb80 1 Processor -- start 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e89e1b67a0 con 0x55e89e02be40 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e89e031b20 con 0x55e89e1aa280 2024-05-10T21:16:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e89e1b6920 con 0x55e89e1aa690 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f07b10dcb80 10 client.0 _ll_register_callbacks cb 0x55e89e05f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60664/0 (socket says 172.21.15.50:60664) 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f079b400700 1 -- 172.21.15.50:0/2237819543 learned_addr learned my addr 172.21.15.50:0/2237819543 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f079aa00700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.800+0000 7f079be00700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 -- 172.21.15.50:0/2237819543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 msgr2=0x55e89e1b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 -- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 msgr2=0x55e89e1b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 -- 172.21.15.50:0/2237819543 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89e091420 con 0x55e89e02be40 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079aa00700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f079b400700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f078c014040 tx=0x7f078c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11c1f91964c5e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.801+0000 7f0785600700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c01e040 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c00f930 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f078c0050d0 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f078c0052d0 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f07b10dcb80 10 client.16458 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e89e1c14a0 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.802+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078c00f930 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f078c021070 con 0x55e89e02be40 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 20 client.16458 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e1c39d0 0x55e89e1c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 4 client.16458 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e89e176070 con 0x55e89e1c39d0 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.803+0000 7f079aa00700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e1c39d0 0x55e89e1c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.804+0000 7f079aa00700 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e1c39d0 0x55e89e1c5df0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b9cc880963b84 server_cookie=b4afa09e99f84fd8 in_seq=0 out_seq=0 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.804+0000 7f0785600700 10 client.16458 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f0785600700 1 -- 172.21.15.50:0/2237819543 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e89e176070 con 0x55e89e1c39d0 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f0785600700 10 client.16458 handle_command_reply: tid=0 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 client.16458 shutdown 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 msgr2=0x55e89e1b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f078c014040 tx=0x7f078c00fd10 comp rx=0 tx=0).stop 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 shutdown_connections 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e1c39d0 0x55e89e1c5df0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b33f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/2237819543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f0785600700 0 client.16458 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 >> 172.21.15.50:0/2237819543 conn(0x55e89e06aa80 msgr2=0x55e89e1a8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.805+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 shutdown_connections 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.806+0000 7f07b10dcb80 1 -- 172.21.15.50:0/2237819543 wait complete. 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.806+0000 7f07b10dcb80 20 client.16458 trim_cache size 0 max 16384 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.822+0000 7f07b10dcb80 1 Processor -- start 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 -- start start 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 0x55e89e1ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b77a0 con 0x55e89e02be40 2024-05-10T21:16:29.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b4fd0 con 0x55e89e1aa690 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b2e80 con 0x55e89e1aa280 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60678/0 (socket says 172.21.15.50:60678) 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f079aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f079b400700 1 -- 172.21.15.50:0/3298623033 learned_addr learned my addr 172.21.15.50:0/3298623033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.823+0000 7f079be00700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 0x55e89e1ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 -- 172.21.15.50:0/3298623033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 msgr2=0x55e89e1b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 -- 172.21.15.50:0/3298623033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 msgr2=0x55e89e1b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 -- 172.21.15.50:0/3298623033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89e1c6290 con 0x55e89e1aa690 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079b400700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079aa00700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.824+0000 7f079be00700 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 0x55e89e1ae6c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07940a7e50 tx=0x7f07940099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd8fd7baa732de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.825+0000 7f0787e00700 1 -- 172.21.15.50:0/3298623033 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940ab2e0 con 0x55e89e1aa690 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.825+0000 7f0787e00700 1 -- 172.21.15.50:0/3298623033 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940aabe0 con 0x55e89e1aa690 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.825+0000 7f0787e00700 1 -- 172.21.15.50:0/3298623033 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794003c00 con 0x55e89e1aa690 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.825+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3298623033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 msgr2=0x55e89e1ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.825+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 0x55e89e1ae6c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07940a7e50 tx=0x7f07940099e0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3298623033 shutdown_connections 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa690 0x55e89e1ae6c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3298623033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa280 0x55e89e1b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3298623033 >> 172.21.15.50:0/3298623033 conn(0x55e89e06aa80 msgr2=0x55e89e1a3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3298623033 shutdown_connections 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3298623033 wait complete. 2024-05-10T21:16:29.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 -- start start 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.826+0000 7f07b10dcb80 1 Processor -- start 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b34d0 con 0x55e89e02be40 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b4900 con 0x55e89e1aa280 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b3b60 con 0x55e89e1aa690 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f07b10dcb80 10 client.0 _ll_register_callbacks cb 0x55e89e05f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f079b400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f079aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f079be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f079aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39804/0 (socket says 172.21.15.50:39804) 2024-05-10T21:16:29.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.827+0000 7f079aa00700 1 -- 172.21.15.50:0/3082864696 learned_addr learned my addr 172.21.15.50:0/3082864696 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 -- 172.21.15.50:0/3082864696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 msgr2=0x55e89e1b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 -- 172.21.15.50:0/3082864696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 msgr2=0x55e89e1bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 -- 172.21.15.50:0/3082864696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89e1b7c50 con 0x55e89e1aa280 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079be00700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f079aa00700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e89e1a6220 tx=0x7f078801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a55c170bee2284 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f0785600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.828+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788004830 con 0x55e89e1aa280 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940aabe0 con 0x55e89e1aa280 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f07880049d0 con 0x55e89e1aa280 2024-05-10T21:16:29.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0788004c10 con 0x55e89e1aa280 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f07b10dcb80 10 client.26612 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e89e22dcc0 con 0x55e89e1aa280 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.829+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940aabe0 con 0x55e89e1aa280 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0788028ce0 con 0x55e89e1aa280 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 10 client.26612 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 20 client.26612 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e22e430 0x55e89e230850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 4 client.26612 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e89e0d9400 con 0x55e89e22e430 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.830+0000 7f079b400700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e22e430 0x55e89e230850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f079b400700 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e22e430 0x55e89e230850 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcf80184c89fe00 server_cookie=f57cd747d843e0c2 in_seq=0 out_seq=0 2024-05-10T21:16:29.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f0785600700 10 client.26612 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f0785600700 1 -- 172.21.15.50:0/3082864696 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e89e0d9400 con 0x55e89e22e430 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f0785600700 10 client.26612 handle_command_reply: tid=0 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 client.26612 shutdown 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 msgr2=0x55e89e1b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e89e1a6220 tx=0x7f078801b000 comp rx=0 tx=0).stop 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 shutdown_connections 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55e89e22e430 0x55e89e230850 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e89e1aa690 0x55e89e1b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f0785600700 0 client.26612 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e89e02be40 0x55e89e1bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 --2- 172.21.15.50:0/3082864696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e89e1aa280 0x55e89e1b71d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 >> 172.21.15.50:0/3082864696 conn(0x55e89e06aa80 msgr2=0x55e89e1b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 shutdown_connections 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 1 -- 172.21.15.50:0/3082864696 wait complete. 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.831+0000 7f07b10dcb80 20 client.26612 trim_cache size 0 max 16384 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 msgr2=0x7f07800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 0x7f07800b4d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f079c116c90 tx=0x7f0788011720 comp rx=0 tx=0).stop 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 msgr2=0x7f079c1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:29.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 0x7f079c1a5a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f079400b070 tx=0x7f0794004d00 comp rx=0 tx=0).stop 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 shutdown_connections 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f07800b2880 0x7f07800b4d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c115180 0x7f079c1a5a20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f079c075b60 0x7f079c1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 --2- 172.21.15.50:0/2584357521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f079c0764b0 0x7f079c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 >> 172.21.15.50:0/2584357521 conn(0x7f079c079d10 msgr2=0x7f079c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 shutdown_connections 2024-05-10T21:16:29.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:29.832+0000 7f07a1000700 1 -- 172.21.15.50:0/2584357521 wait complete. 2024-05-10T21:16:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:30 smithi050 ceph-mon[29546]: pgmap v690: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-10T21:16:31.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:30 smithi092 ceph-mon[34415]: pgmap v690: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-10T21:16:31.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:30 smithi142 ceph-mon[34846]: pgmap v690: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-10T21:16:31.884 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:16:31.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:16:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 -- 172.21.15.50:0/875978438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 msgr2=0x7f746410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 --2- 172.21.15.50:0/875978438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f746410bfd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ca0 tx=0x7f745c02a8c0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 -- 172.21.15.50:0/875978438 shutdown_connections 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 --2- 172.21.15.50:0/875978438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f746410bfd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 --2- 172.21.15.50:0/875978438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f74641052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 --2- 172.21.15.50:0/875978438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 -- 172.21.15.50:0/875978438 >> 172.21.15.50:0/875978438 conn(0x7f74640fee60 msgr2=0x7f7464101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 -- 172.21.15.50:0/875978438 shutdown_connections 2024-05-10T21:16:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.590+0000 7f746bc00640 1 -- 172.21.15.50:0/875978438 wait complete. 2024-05-10T21:16:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.591+0000 7f746bc00640 1 Processor -- start 2024-05-10T21:16:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 -- start start 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f7464198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f7464195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f746410f7f0 con 0x7f7464104eb0 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f746410f670 con 0x7f7464103c70 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f746bc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f746410f970 con 0x7f7464109bb0 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f7469200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.592+0000 7f7469200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39820/0 (socket says 172.21.15.50:39820) 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7469200640 1 -- 172.21.15.50:0/816801177 learned_addr learned my addr 172.21.15.50:0/816801177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f7464198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7469c00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f7464195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 -- 172.21.15.50:0/816801177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 msgr2=0x7f7464195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f7464195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 -- 172.21.15.50:0/816801177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 msgr2=0x7f7464198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 -- 172.21.15.50:0/816801177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7464195a70 con 0x7f7464104eb0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7469c00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f7464195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.593+0000 7f7463e00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f7464198c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9f0 tx=0x7f745000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.594+0000 7f7461600640 1 -- 172.21.15.50:0/816801177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000ccc0 con 0x7f7464104eb0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.594+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7464195d60 con 0x7f7464104eb0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.594+0000 7f7461600640 1 -- 172.21.15.50:0/816801177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500108e0 con 0x7f7464104eb0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.594+0000 7f7461600640 1 -- 172.21.15.50:0/816801177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450018880 con 0x7f7464104eb0 2024-05-10T21:16:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.594+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a8cc0 con 0x7f7464104eb0 2024-05-10T21:16:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.598+0000 7f7472b7c740 1 Processor -- start 2024-05-10T21:16:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.598+0000 7f7472b7c740 1 -- start start 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56295f956550 con 0x56295f6272f0 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56295f8fbbc0 con 0x56295f62f4d0 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7472b7c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56295f952510 con 0x56295f6501b0 2024-05-10T21:16:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7469c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7469200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7469c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35148/0 (socket says 172.21.15.50:35148) 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.599+0000 7f7469c00640 1 -- 172.21.15.50:0/3419449059 learned_addr learned my addr 172.21.15.50:0/3419449059 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 -- 172.21.15.50:0/3419449059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 msgr2=0x56295f9f9580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9f9580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 -- 172.21.15.50:0/3419449059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 msgr2=0x56295f9f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 -- 172.21.15.50:0/3419449059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295f668210 con 0x56295f6501b0 2024-05-10T21:16:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7461600640 1 -- 172.21.15.50:0/816801177 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7450010430 con 0x7f7464104eb0 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.600+0000 7f7469c00640 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f745c035000 tx=0x7f745c02f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f631a8f393d33f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.601+0000 7f7456000640 1 -- 172.21.15.50:0/3419449059 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c031650 con 0x56295f6501b0 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.601+0000 7f7456000640 1 -- 172.21.15.50:0/3419449059 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745c036040 con 0x56295f6501b0 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.601+0000 7f7456000640 1 -- 172.21.15.50:0/3419449059 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c0045f0 con 0x56295f6501b0 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.601+0000 7f7472b7c740 1 -- 172.21.15.50:0/3419449059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 msgr2=0x56295f9f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f745c035000 tx=0x7f745c02f450 comp rx=0 tx=0).stop 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 -- 172.21.15.50:0/3419449059 shutdown_connections 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 --2- 172.21.15.50:0/3419449059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295f9f2100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 -- 172.21.15.50:0/3419449059 >> 172.21.15.50:0/3419449059 conn(0x56295f644300 msgr2=0x56295f9eda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 -- 172.21.15.50:0/3419449059 shutdown_connections 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7461600640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 0x7f744c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 -- 172.21.15.50:0/3419449059 wait complete. 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7461600640 1 -- 172.21.15.50:0/816801177 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f74500e2540 con 0x7f7464104eb0 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7469200640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 0x7f744c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.602+0000 7f7472b7c740 1 -- start start 2024-05-10T21:16:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 Processor -- start 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7469200640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 0x7f744c0b4cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56295f9ee820 tx=0x7f745800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 0x56295f9fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 0x56295fa02ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56295f9f9df0 con 0x56295f62f4d0 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56295f90f660 con 0x56295f6501b0 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56295f9f9f70 con 0x56295f6272f0 2024-05-10T21:16:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.603+0000 7f7472b7c740 10 client.0 _ll_register_callbacks cb 0x56295f8d9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 0x56295f9fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7469200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7469200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9fba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35158/0 (socket says 172.21.15.50:35158) 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7469200640 1 -- 172.21.15.50:0/275199917 learned_addr learned my addr 172.21.15.50:0/275199917 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7469c00640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 0x56295fa02ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 -- 172.21.15.50:0/275199917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 msgr2=0x56295f9fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9fba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 -- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 msgr2=0x56295fa02ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 0x56295fa02ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7463e00640 1 -- 172.21.15.50:0/275199917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295f734620 con 0x56295f62f4d0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.604+0000 7f7469c00640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 0x56295fa02ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f7463e00640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 0x56295f9fbf80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f74500e0050 tx=0x7f74500b23d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f3cc534e083545 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f744aa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500adf20 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f74500e20e0 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c036040 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74500b29d0 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f7472b7c740 10 client.16476 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.605+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56295fa05e80 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745c036040 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f74500a5720 con 0x56295f62f4d0 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 10 client.16476 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 20 client.16476 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f60ed80 0x56295fa094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 4 client.16476 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.606+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56295f964160 con 0x56295f60ed80 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.607+0000 7f7469200640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f60ed80 0x56295fa094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.608+0000 7f744aa00640 10 client.16476 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.608+0000 7f7469200640 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f60ed80 0x56295fa094c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819ba8c69e5ead5e server_cookie=695ef8258e709fab in_seq=0 out_seq=0 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f744aa00640 1 -- 172.21.15.50:0/275199917 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56295f964160 con 0x56295f60ed80 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f744aa00640 10 client.16476 handle_command_reply: tid=0 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 client.16476 shutdown 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 msgr2=0x56295f9fbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 0x56295f9fbf80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f74500e0050 tx=0x7f74500b23d0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 shutdown_connections 2024-05-10T21:16:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f60ed80 0x56295fa094c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f6501b0 0x56295fa02ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f62f4d0 0x56295f9fbf80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 --2- 172.21.15.50:0/275199917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f744aa00640 0 client.16476 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 >> 172.21.15.50:0/275199917 conn(0x56295f644300 msgr2=0x56295f9eca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.609+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 shutdown_connections 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.610+0000 7f7472b7c740 1 -- 172.21.15.50:0/275199917 wait complete. 2024-05-10T21:16:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.610+0000 7f7472b7c740 20 client.16476 trim_cache size 0 max 16384 2024-05-10T21:16:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 Processor -- start 2024-05-10T21:16:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 -- start start 2024-05-10T21:16:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295fa0dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74641b4120 con 0x56295f6272f0 2024-05-10T21:16:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74641b1990 con 0x56295f62f4d0 2024-05-10T21:16:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7472b7c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74641af7e0 con 0x56295f6501b0 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7469200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.627+0000 7f7469200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9fa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60746/0 (socket says 172.21.15.50:60746) 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7469200640 1 -- 172.21.15.50:0/423539790 learned_addr learned my addr 172.21.15.50:0/423539790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7469c00640 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295fa0dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 -- 172.21.15.50:0/423539790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 msgr2=0x56295fa0dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295fa0dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 -- 172.21.15.50:0/423539790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 msgr2=0x56295f9fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 -- 172.21.15.50:0/423539790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295f9fa0f0 con 0x56295f62f4d0 2024-05-10T21:16:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7463e00640 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9fa8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74500a8e10 tx=0x7f74500a5c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552ea206fc65c8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.628+0000 7f7455600640 1 -- 172.21.15.50:0/423539790 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500adf20 con 0x56295f62f4d0 2024-05-10T21:16:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.629+0000 7f7455600640 1 -- 172.21.15.50:0/423539790 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500c41a0 con 0x56295f62f4d0 2024-05-10T21:16:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.629+0000 7f7455600640 1 -- 172.21.15.50:0/423539790 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500a57f0 con 0x56295f62f4d0 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- 172.21.15.50:0/423539790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 msgr2=0x56295f9fa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9fa8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74500a8e10 tx=0x7f74500a5c10 comp rx=0 tx=0).stop 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- 172.21.15.50:0/423539790 shutdown_connections 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6501b0 0x56295fa0dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9fa8b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 --2- 172.21.15.50:0/423539790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6272f0 0x56295f9fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- 172.21.15.50:0/423539790 >> 172.21.15.50:0/423539790 conn(0x56295f644300 msgr2=0x56295f9f6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- 172.21.15.50:0/423539790 shutdown_connections 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- 172.21.15.50:0/423539790 wait complete. 2024-05-10T21:16:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 -- start start 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.630+0000 7f7472b7c740 1 Processor -- start 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 0x56295fa6f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74641afe50 con 0x56295f6501b0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74641b12c0 con 0x56295f62f4d0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74641b0520 con 0x56295f6272f0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7472b7c740 10 client.0 _ll_register_callbacks cb 0x7f74500a5b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7469c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 0x56295fa6f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7463e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7463e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:39850/0 (socket says 172.21.15.50:39850) 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7463e00640 1 -- 172.21.15.50:0/2271017999 learned_addr learned my addr 172.21.15.50:0/2271017999 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7469c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 0x56295fa6f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60752/0 (socket says 172.21.15.50:60752) 2024-05-10T21:16:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.631+0000 7f7469200640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 -- 172.21.15.50:0/2271017999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 msgr2=0x56295f9ff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9ff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 -- 172.21.15.50:0/2271017999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 msgr2=0x56295fa6f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 0x56295fa6f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 -- 172.21.15.50:0/2271017999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295fa6fda0 con 0x56295f62f4d0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7463e00640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f74500c0050 tx=0x7f74500a5840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd47043b4db51f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f744aa00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500a91b0 con 0x56295f62f4d0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f74500af0b0 con 0x56295f62f4d0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.632+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74500c41a0 con 0x56295f62f4d0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74500adf20 con 0x56295f62f4d0 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 10 client.26627 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56295fa70f30 con 0x56295f62f4d0 2024-05-10T21:16:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74500c41a0 con 0x56295f62f4d0 2024-05-10T21:16:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7450018b40 con 0x56295f62f4d0 2024-05-10T21:16:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 10 client.26627 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 20 client.26627 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f5e41c0 0x56295fa73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 4 client.26627 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.633+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56295f95dfd0 con 0x56295f5e41c0 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.634+0000 7f7469200640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f5e41c0 0x56295fa73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.634+0000 7f744aa00640 10 client.26627 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.634+0000 7f7469200640 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f5e41c0 0x56295fa73680 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8315a2a7f71867 server_cookie=fd69fd89fd64bdf7 in_seq=0 out_seq=0 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.904+0000 7f744aa00640 1 -- 172.21.15.50:0/2271017999 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56295f95dfd0 con 0x56295f5e41c0 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f744aa00640 10 client.26627 handle_command_reply: tid=0 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 client.26627 shutdown 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 msgr2=0x56295f9ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f74500c0050 tx=0x7f74500a5840 comp rx=0 tx=0).stop 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 shutdown_connections 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56295f5e41c0 0x56295fa73680 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56295f6501b0 0x56295fa6f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56295f62f4d0 0x56295f9ff9b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 --2- 172.21.15.50:0/2271017999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56295f6272f0 0x56295f9ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f744aa00640 0 client.26627 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 >> 172.21.15.50:0/2271017999 conn(0x56295f644300 msgr2=0x56295f9fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 shutdown_connections 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.905+0000 7f7472b7c740 1 -- 172.21.15.50:0/2271017999 wait complete. 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.906+0000 7f7472b7c740 20 client.26627 trim_cache size 0 max 16384 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:16:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:16:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:16:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 msgr2=0x7f744c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 0x7f744c0b4cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56295f9ee820 tx=0x7f745800b410 comp rx=0 tx=0).stop 2024-05-10T21:16:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 msgr2=0x7f7464198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f7464198c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9f0 tx=0x7f745000eeb0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 shutdown_connections 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f744c0b27e0 0x7f744c0b4cf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7464109bb0 0x7f7464195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7464104eb0 0x7f7464198c40 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f745000e9f0 tx=0x7f745000eeb0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 --2- 172.21.15.50:0/816801177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7464103c70 0x7f7464198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.907+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 >> 172.21.15.50:0/816801177 conn(0x7f74640fee60 msgr2=0x7f74640ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.908+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 shutdown_connections 2024-05-10T21:16:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:32.908+0000 7f746bc00640 1 -- 172.21.15.50:0/816801177 wait complete. 2024-05-10T21:16:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:32 smithi050 ceph-mon[29546]: pgmap v691: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-10T21:16:33.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:32 smithi092 ceph-mon[34415]: pgmap v691: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-10T21:16:33.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:32 smithi142 ceph-mon[34846]: pgmap v691: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-10T21:16:34.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 -- 172.21.15.50:0/2235728968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048075b60 msgr2=0x7fb048075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 --2- 172.21.15.50:0/2235728968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048075b60 0x7fb048075f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 -- 172.21.15.50:0/2235728968 shutdown_connections 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 --2- 172.21.15.50:0/2235728968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048115370 0x7fb048117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 --2- 172.21.15.50:0/2235728968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048075b60 0x7fb048075f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.647+0000 7fb04f600700 1 --2- 172.21.15.50:0/2235728968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb048114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.648+0000 7fb04f600700 1 -- 172.21.15.50:0/2235728968 >> 172.21.15.50:0/2235728968 conn(0x7fb048100040 msgr2=0x7fb048102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.648+0000 7fb04f600700 1 -- 172.21.15.50:0/2235728968 shutdown_connections 2024-05-10T21:16:34.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.648+0000 7fb04f600700 1 -- 172.21.15.50:0/2235728968 wait complete. 2024-05-10T21:16:34.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.649+0000 7fb04f600700 1 Processor -- start 2024-05-10T21:16:34.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.649+0000 7fb04f600700 1 -- start start 2024-05-10T21:16:34.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.649+0000 7fb04f600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 0x7fb0481a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.649+0000 7fb04f600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04f600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb04811aad0 con 0x7fb0480764b0 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04f600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb04811a950 con 0x7fb048075b60 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb04811ac50 con 0x7fb048115370 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb047e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 0x7fb0481a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48480/0 (socket says 172.21.15.50:48480) 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb047e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52418/0 (socket says 172.21.15.50:52418) 2024-05-10T21:16:34.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.650+0000 7fb04d600700 1 -- 172.21.15.50:0/1873312919 learned_addr learned my addr 172.21.15.50:0/1873312919 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 -- 172.21.15.50:0/1873312919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 msgr2=0x7fb0481a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 0x7fb0481a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 -- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 msgr2=0x7fb0481a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 -- 172.21.15.50:0/1873312919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0481a6370 con 0x7fb048115370 2024-05-10T21:16:34.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04cc00700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 0x7fb0481a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb047e00700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04d600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c370 tx=0x7fb04000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb045600700 1 -- 172.21.15.50:0/1873312919 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400092e0 con 0x7fb048115370 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb045600700 1 -- 172.21.15.50:0/1873312919 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb040003a00 con 0x7fb048115370 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0481a6600 con 0x7fb048115370 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.651+0000 7fb045600700 1 -- 172.21.15.50:0/1873312919 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400046f0 con 0x7fb048115370 2024-05-10T21:16:34.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.652+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0481adf80 con 0x7fb048115370 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.655+0000 7fb045600700 1 -- 172.21.15.50:0/1873312919 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb04001a020 con 0x7fb048115370 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.656+0000 7fb05e18bb80 1 Processor -- start 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 -- start start 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb045600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 0x7fb0300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 0x564141507790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb045600700 1 -- 172.21.15.50:0/1873312919 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb0400e19d0 con 0x7fb048115370 2024-05-10T21:16:34.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 0x5641415080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564141429aa0 con 0x564141389ec0 2024-05-10T21:16:34.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5641413c77e0 con 0x56414150dc70 2024-05-10T21:16:34.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.657+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564141425a60 con 0x564141507cd0 2024-05-10T21:16:34.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52432/0 (socket says 172.21.15.50:52432) 2024-05-10T21:16:34.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 -- 172.21.15.50:0/2621046326 learned_addr learned my addr 172.21.15.50:0/2621046326 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb047e00700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 0x564141507790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb047e00700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 0x7fb0300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04d600700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 0x5641415080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 -- 172.21.15.50:0/2621046326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 msgr2=0x5641415080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 0x5641415080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 -- 172.21.15.50:0/2621046326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 msgr2=0x564141507790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 0x564141507790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04cc00700 1 -- 172.21.15.50:0/2621046326 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564141376950 con 0x564141389ec0 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb04d600700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 0x5641415080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.658+0000 7fb047e00700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 0x564141507790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.659+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0096a0 tx=0x7fb03c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e5baf56d146713 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.659+0000 7fb047e00700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 0x7fb0300b4e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb034005950 tx=0x7fb034009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.659+0000 7fb03a000700 1 -- 172.21.15.50:0/2621046326 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c005270 con 0x564141389ec0 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.659+0000 7fb03a000700 1 -- 172.21.15.50:0/2621046326 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c005410 con 0x564141389ec0 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.659+0000 7fb03a000700 1 -- 172.21.15.50:0/2621046326 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00f750 con 0x564141389ec0 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2621046326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 msgr2=0x56414150d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0096a0 tx=0x7fb03c0096d0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2621046326 shutdown_connections 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141507cd0 0x5641415080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564141389ec0 0x56414150d730 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2621046326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56414150dc70 0x564141507790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.660+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2621046326 >> 172.21.15.50:0/2621046326 conn(0x5641413c8b00 msgr2=0x564141502810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.661+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2621046326 shutdown_connections 2024-05-10T21:16:34.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.661+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2621046326 wait complete. 2024-05-10T21:16:34.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.662+0000 7fb05e18bb80 1 -- start start 2024-05-10T21:16:34.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.662+0000 7fb05e18bb80 1 Processor -- start 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141511400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564141514680 con 0x56414150dc70 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56414138fba0 con 0x564141507cd0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564141514800 con 0x564141389ec0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb05e18bb80 10 client.0 _ll_register_callbacks cb 0x5641413bd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37634/0 (socket says 172.21.15.50:37634) 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb047e00700 1 -- 172.21.15.50:0/2291072117 learned_addr learned my addr 172.21.15.50:0/2291072117 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141511400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.663+0000 7fb04d600700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 -- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 msgr2=0x564141511940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 -- 172.21.15.50:0/2291072117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 msgr2=0x564141511e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 -- 172.21.15.50:0/2291072117 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641413ef4a0 con 0x564141389ec0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb047e00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141511400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe473ff2f655ac34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.664+0000 7fb02ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c02f020 con 0x564141389ec0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb03c003890 con 0x564141389ec0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03c005410 con 0x564141389ec0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb03c003a90 con 0x564141389ec0 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb05e18bb80 10 client.26584 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:34.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.665+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564141514680 con 0x564141389ec0 2024-05-10T21:16:34.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c005410 con 0x564141389ec0 2024-05-10T21:16:34.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb03c022430 con 0x564141389ec0 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb05e18bb80 10 client.26584 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb05e18bb80 10 client.26584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb05e18bb80 20 client.26584 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.666+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5641415214f0 0x564141523910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.667+0000 7fb05e18bb80 4 client.26584 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:34.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.667+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641414d40f0 con 0x5641415214f0 2024-05-10T21:16:34.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.667+0000 7fb047e00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5641415214f0 0x564141523910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.667+0000 7fb047e00700 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5641415214f0 0x564141523910 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9da8ca36f2813c6 server_cookie=9643fa8838dcaaa3 in_seq=0 out_seq=0 2024-05-10T21:16:34.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.668+0000 7fb02ea00700 10 client.26584 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:34.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.668+0000 7fb02ea00700 1 -- 172.21.15.50:0/2291072117 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641414d40f0 con 0x5641415214f0 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.668+0000 7fb02ea00700 10 client.26584 handle_command_reply: tid=0 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.668+0000 7fb05e18bb80 1 client.26584 shutdown 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 msgr2=0x564141511400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141511400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c005280 comp rx=0 tx=0).stop 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 shutdown_connections 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5641415214f0 0x564141523910 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141511400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2291072117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141511940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 >> 172.21.15.50:0/2291072117 conn(0x5641413c8b00 msgr2=0x564141506780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb02ea00700 0 client.26584 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 shutdown_connections 2024-05-10T21:16:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2291072117 wait complete. 2024-05-10T21:16:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.669+0000 7fb05e18bb80 20 client.26584 trim_cache size 0 max 16384 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.692+0000 7fb05e18bb80 1 Processor -- start 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.692+0000 7fb05e18bb80 1 -- start start 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141507790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b78a0 con 0x56414150dc70 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b50d0 con 0x564141507cd0 2024-05-10T21:16:34.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b2f60 con 0x564141389ec0 2024-05-10T21:16:34.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb04d600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37644/0 (socket says 172.21.15.50:37644) 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb047e00700 1 -- 172.21.15.50:0/1726065826 learned_addr learned my addr 172.21.15.50:0/1726065826 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.693+0000 7fb04cc00700 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141507790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 -- 172.21.15.50:0/1726065826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 msgr2=0x564141507790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141507790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 -- 172.21.15.50:0/1726065826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 msgr2=0x564141511cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 -- 172.21.15.50:0/1726065826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414150f990 con 0x564141507cd0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb047e00700 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x564141504010 tx=0x7fb0340058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b022c0eeb31a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb039600700 1 -- 172.21.15.50:0/1726065826 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03401cc70 con 0x564141507cd0 2024-05-10T21:16:34.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.694+0000 7fb039600700 1 -- 172.21.15.50:0/1726065826 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03401ce10 con 0x564141507cd0 2024-05-10T21:16:34.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.695+0000 7fb039600700 1 -- 172.21.15.50:0/1726065826 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340055b0 con 0x564141507cd0 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- 172.21.15.50:0/1726065826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 msgr2=0x564141502940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x564141504010 tx=0x7fb0340058e0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- 172.21.15.50:0/1726065826 shutdown_connections 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141511cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141507790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/1726065826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141502940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- 172.21.15.50:0/1726065826 >> 172.21.15.50:0/1726065826 conn(0x5641413c8b00 msgr2=0x56414150b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- 172.21.15.50:0/1726065826 shutdown_connections 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- 172.21.15.50:0/1726065826 wait complete. 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.696+0000 7fb05e18bb80 1 -- start start 2024-05-10T21:16:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 Processor -- start 2024-05-10T21:16:34.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141519d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141516fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b35b0 con 0x56414150dc70 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b4a00 con 0x564141507cd0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481b3c60 con 0x564141389ec0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb05e18bb80 10 client.0 _ll_register_callbacks cb 0x5641413bd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.697+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37652/0 (socket says 172.21.15.50:37652) 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 -- 172.21.15.50:0/2733548305 learned_addr learned my addr 172.21.15.50:0/2733548305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141519d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb04d600700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141516fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 -- 172.21.15.50:0/2733548305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 msgr2=0x564141519d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141519d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 -- 172.21.15.50:0/2733548305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 msgr2=0x564141516fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141516fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 -- 172.21.15.50:0/2733548305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564141517510 con 0x564141507cd0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb047e00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e00 tx=0x7fb034009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbdbfa907ed1c0de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.698+0000 7fb02ea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03403a060 con 0x564141507cd0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb034005440 con 0x564141507cd0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03401ce10 con 0x564141507cd0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb03401cc20 con 0x564141507cd0 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb05e18bb80 10 client.26645 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56414158c750 con 0x564141507cd0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.699+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03401ce10 con 0x564141507cd0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb03401b070 con 0x564141507cd0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 10 client.26645 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 20 client.26645 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56414158cde0 0x56414158f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 4 client.26645 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564141437480 con 0x56414158cde0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.700+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56414158cde0 0x56414158f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.701+0000 7fb04cc00700 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56414158cde0 0x56414158f200 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c200d5a601950b79 server_cookie=97940cab9349d319 in_seq=0 out_seq=0 2024-05-10T21:16:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.701+0000 7fb02ea00700 10 client.26645 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb02ea00700 1 -- 172.21.15.50:0/2733548305 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564141437480 con 0x56414158cde0 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb02ea00700 10 client.26645 handle_command_reply: tid=0 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 client.26645 shutdown 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 msgr2=0x564141516a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e00 tx=0x7fb034009c60 comp rx=0 tx=0).stop 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 shutdown_connections 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56414158cde0 0x56414158f200 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56414150dc70 0x564141516fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564141389ec0 0x564141519d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb05e18bb80 1 --2- 172.21.15.50:0/2733548305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564141507cd0 0x564141516a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.702+0000 7fb02ea00700 0 client.26645 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.703+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 >> 172.21.15.50:0/2733548305 conn(0x5641413c8b00 msgr2=0x564141506780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.703+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 shutdown_connections 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.703+0000 7fb05e18bb80 1 -- 172.21.15.50:0/2733548305 wait complete. 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.703+0000 7fb05e18bb80 20 client.26645 trim_cache size 0 max 16384 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "f7e7dc4b-3d4a-4b30-b57a-7a8435e9b4d2", 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:16:34.701 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 msgr2=0x7fb0300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 0x7fb0300b4e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb034005950 tx=0x7fb034009260 comp rx=0 tx=0).stop 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 msgr2=0x7fb0481a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c370 tx=0x7fb04000c680 comp rx=0 tx=0).stop 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 shutdown_connections 2024-05-10T21:16:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb0300b2900 0x7fb0300b4e10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb048075b60 0x7fb0481a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb048115370 0x7fb0481a5cd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 --2- 172.21.15.50:0/1873312919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0480764b0 0x7fb0481a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 >> 172.21.15.50:0/1873312919 conn(0x7fb048100040 msgr2=0x7fb048100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.704+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 shutdown_connections 2024-05-10T21:16:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:34.705+0000 7fb04f600700 1 -- 172.21.15.50:0/1873312919 wait complete. 2024-05-10T21:16:35.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:34 smithi092 ceph-mon[34415]: pgmap v692: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-05-10T21:16:35.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:34 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:16:35.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:34 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000001518 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40) see mds.i log and `damage ls` output for details 2024-05-10T21:16:35.115 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:16:34 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: 2024-05-10T21:16:34.847+0000 7fd1e137f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000e24 [...2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33/ auth{1=1,2=1} v128405 ap=2 f(v4 m2024-05-10T21:16:17.167064+0000 59=59+0) n(v7 rc2024-05-10T21:16:17.167064+0000 b1936351 60=59+1) (inest sync->mix g=1,2) caps={14925=pAsLsXsFs/pAsLsXsFsx@4} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5575805cbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000000e24:[<0x10000000399/blog-33 v118010>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(2)0x20000000e24:[<0x10000000399/blog-33 v128405>,<0x10000000208/blogtest_in v2122>,<0x10000000201/src v1786>,<0x10000000006/blogbench-1.0 v1567>,<0x10000000005/tmp v134>,<0x100000001fb/client.0 v135>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v147>,<0x10000000001/sv_1 v171>,<0x10000000000/qa v164>,<0x1/volumes v153>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T21:15:56.972560+0000 58=58+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T21:16:17.167064+0000 b1936351 60=59+1)","memory_value.dirstat":"f(v4 m2024-05-10T21:16:17.167064+0000 59=59+0)","memory_value.rstat":"n(v7 rc2024-05-10T21:16:17.167064+0000 b1936351 60=59+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T21:16:35.139 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:16:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:16:34.875+0000 7f885a200700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000bd9 [...2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29/ auth{0=1,2=1} fragtree_t(*^3) v88943 ap=2 DIRTYPARENT f(v4 m2024-05-10T21:13:29.828217+0000 134=134+0) n(v6 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1) (inest sync->mix g=0,2) caps={14925=pAsLsXsFs/pAsLsXsFsx@161} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55ff492d8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000000bd9:[<0x10000000399/blog-29 v88943>,<0x10000000208/blogtest_in v2124>,<0x10000000201/src v1786>,<0x10000000006/blogbench-1.0 v1549>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v117>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v129>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v117>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T21:12:38.539504+0000 133=133+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1)","memory_value.dirstat":"f(v4 m2024-05-10T21:13:29.828217+0000 134=134+0)","memory_value.rstat":"n(v6 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-05-10T21:16:35.139 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:16:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:16:35.043+0000 7f885a200700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000d5b [...2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-32/ auth{0=1,2=1} v113368 ap=2 DIRTYPARENT f(v5 m2024-05-10T21:15:35.286095+0000 70=70+0) n(v7 rc2024-05-10T21:15:35.286095+0000 b1768554 71=70+1) (inest sync->mix g=0,2) caps={14925=pAsLsXsFs/pAsLsXsFsx@170} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55ff48744680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000000d5b:[<0x10000000399/blog-32 v113368>,<0x10000000208/blogtest_in v2126>,<0x10000000201/src v1786>,<0x10000000006/blogbench-1.0 v1549>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v117>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v129>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v117>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T21:15:26.871304+0000 69=69+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T21:15:35.286095+0000 b1768554 71=70+1)","memory_value.dirstat":"f(v5 m2024-05-10T21:15:35.286095+0000 70=70+0)","memory_value.rstat":"n(v7 rc2024-05-10T21:15:35.286095+0000 b1768554 71=70+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-10T21:16:35.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:34 smithi050 ceph-mon[29546]: pgmap v692: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-05-10T21:16:35.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:34 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:16:35.140 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:34 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000001518 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40) see mds.i log and `damage ls` output for details 2024-05-10T21:16:35.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:34 smithi142 ceph-mon[34846]: pgmap v692: 97 pgs: 97 active+clean; 615 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-05-10T21:16:35.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:34 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:16:35.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:34 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000001518 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40) see mds.i log and `damage ls` output for details 2024-05-10T21:16:35.376 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:16:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: 2024-05-10T21:16:34.928+0000 7f0648b5d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001518 [...2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40/ auth{0=1} v94279 ap=2 DIRTYPARENT f(v3 m2024-05-10T21:15:24.004375+0000 100=100+0) n(v8 rc2024-05-10T21:15:24.004375+0000 b3301671 101=100+1) (inest sync->mix g=0) caps={14925=pAsLsXsFs/pAsLsXsFsx@1} mcw={0=AsLsXsFs} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyparent=1 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559894f66680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000001518:[<0x10000000399/blog-40 v94279>,<0x10000000208/blogtest_in v2122>,<0x10000000201/src v1786>,<0x10000000006/blogbench-1.0 v1549>,<0x10000000005/tmp v116>,<0x100000001fb/client.0 v117>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v129>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v117>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T21:13:34.543099+0000 97=97+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T21:15:24.004375+0000 b3301671 101=100+1)","memory_value.dirstat":"f(v3 m2024-05-10T21:15:24.004375+0000 100=100+0)","memory_value.rstat":"n(v8 rc2024-05-10T21:15:24.004375+0000 b3301671 101=100+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-05-10T21:16:35.688 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:16:36.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:16:36.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:36.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:16:36.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:16:36.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000000e24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-05-10T21:16:36.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:16:36.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000000d5b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-32) see mds.a log and `damage ls` output for details 2024-05-10T21:16:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000000e24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:16:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000000d5b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-32) see mds.a log and `damage ls` output for details 2024-05-10T21:16:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:16:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:16:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:16:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000000e24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-05-10T21:16:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:16:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000000d5b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-32) see mds.a log and `damage ls` output for details 2024-05-10T21:16:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:37 smithi092 ceph-mon[34415]: pgmap v693: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 760 op/s 2024-05-10T21:16:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:37 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.367 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:37 smithi050 ceph-mon[29546]: pgmap v693: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 760 op/s 2024-05-10T21:16:37.367 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.367 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.368 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:37 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:37 smithi142 ceph-mon[34846]: pgmap v693: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 760 op/s 2024-05-10T21:16:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:37 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 -- 172.21.15.50:0/3622160695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 msgr2=0x7f91a4106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 --2- 172.21.15.50:0/3622160695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a4106990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9190009990 tx=0x7f9190009ca0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 -- 172.21.15.50:0/3622160695 shutdown_connections 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 --2- 172.21.15.50:0/3622160695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a4106ed0 0x7f91a4113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 --2- 172.21.15.50:0/3622160695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a410c5c0 0x7f91a410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 --2- 172.21.15.50:0/3622160695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a4106990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 -- 172.21.15.50:0/3622160695 >> 172.21.15.50:0/3622160695 conn(0x7f91a4102080 msgr2=0x7f91a41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 -- 172.21.15.50:0/3622160695 shutdown_connections 2024-05-10T21:16:38.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.962+0000 7f91ab000700 1 -- 172.21.15.50:0/3622160695 wait complete. 2024-05-10T21:16:38.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.963+0000 7f91ab000700 1 Processor -- start 2024-05-10T21:16:38.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.963+0000 7f91ab000700 1 -- start start 2024-05-10T21:16:38.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a4106ed0 0x7f91a419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41166a0 con 0x7f91a4106ed0 2024-05-10T21:16:38.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4116520 con 0x7f91a410c5c0 2024-05-10T21:16:38.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91ab000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4116820 con 0x7f91a4106580 2024-05-10T21:16:38.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a4106ed0 0x7f91a419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91a9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91a9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37682/0 (socket says 172.21.15.50:37682) 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.964+0000 7f91a9000700 1 -- 172.21.15.50:0/3551037406 learned_addr learned my addr 172.21.15.50:0/3551037406 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 -- 172.21.15.50:0/3551037406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 msgr2=0x7f91a419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 -- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a4106ed0 msgr2=0x7f91a419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a4106ed0 0x7f91a419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 -- 172.21.15.50:0/3551037406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41a1fa0 con 0x7f91a410c5c0 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a3e00700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:38.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.965+0000 7f91a9000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f919c007ee0 tx=0x7f919c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.966+0000 7f91a0c00700 1 -- 172.21.15.50:0/3551037406 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c004bb0 con 0x7f91a410c5c0 2024-05-10T21:16:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.966+0000 7f91a0c00700 1 -- 172.21.15.50:0/3551037406 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c004d50 con 0x7f91a410c5c0 2024-05-10T21:16:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.966+0000 7f91a0c00700 1 -- 172.21.15.50:0/3551037406 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c005920 con 0x7f91a410c5c0 2024-05-10T21:16:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.966+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41a2290 con 0x7f91a410c5c0 2024-05-10T21:16:38.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.966+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a9bb0 con 0x7f91a410c5c0 2024-05-10T21:16:38.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.970+0000 7f91a0c00700 1 -- 172.21.15.50:0/3551037406 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f919c01a020 con 0x7f91a410c5c0 2024-05-10T21:16:38.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.971+0000 7f91b9b97b80 1 Processor -- start 2024-05-10T21:16:38.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.971+0000 7f91a0c00700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 0x7f91880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 -- start start 2024-05-10T21:16:38.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91a0c00700 1 -- 172.21.15.50:0/3551037406 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f919c0e18f0 con 0x7f91a410c5c0 2024-05-10T21:16:38.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91a3400700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 0x7f91880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55af4c40aa20 con 0x55af4c4e89f0 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55af4c3a8760 con 0x55af4c36ae40 2024-05-10T21:16:38.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.972+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55af4c4069e0 con 0x55af4c4ee830 2024-05-10T21:16:38.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a9000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48544/0 (socket says 172.21.15.50:48544) 2024-05-10T21:16:38.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3400700 1 -- 172.21.15.50:0/1597337069 learned_addr learned my addr 172.21.15.50:0/1597337069 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:37696/0 (socket says 172.21.15.50:37696) 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 -- 172.21.15.50:0/1597337069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 msgr2=0x55af4c4e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 -- 172.21.15.50:0/1597337069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 msgr2=0x55af4c4e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4e8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3e00700 1 -- 172.21.15.50:0/1597337069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4c3578d0 con 0x55af4c36ae40 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.973+0000 7f91a3400700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 0x7f91880b4d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9190009750 tx=0x7f9190005330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.974+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9194009fb0 tx=0x7f9194011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642ffa8a8314fa8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.974+0000 7f9199600700 1 -- 172.21.15.50:0/1597337069 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194004230 con 0x55af4c36ae40 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.974+0000 7f9199600700 1 -- 172.21.15.50:0/1597337069 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004ce0 con 0x55af4c36ae40 2024-05-10T21:16:38.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.974+0000 7f9199600700 1 -- 172.21.15.50:0/1597337069 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194005660 con 0x55af4c36ae40 2024-05-10T21:16:38.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1597337069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 msgr2=0x55af4c4ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:38.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9194009fb0 tx=0x7f9194011720 comp rx=0 tx=0).stop 2024-05-10T21:16:38.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1597337069 shutdown_connections 2024-05-10T21:16:38.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c36ae40 0x55af4c4ee2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1597337069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1597337069 >> 172.21.15.50:0/1597337069 conn(0x55af4c3aff70 msgr2=0x55af4c4e32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1597337069 shutdown_connections 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.975+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1597337069 wait complete. 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.976+0000 7f91b9b97b80 1 -- start start 2024-05-10T21:16:38.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.976+0000 7f91b9b97b80 1 Processor -- start 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 0x55af4c4f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55af4c4f5310 con 0x55af4c36ae40 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55af4c370b20 con 0x55af4c4e89f0 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55af4c4f5490 con 0x55af4c4ee830 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91b9b97b80 10 client.0 _ll_register_callbacks cb 0x55af4c3b1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52492/0 (socket says 172.21.15.50:52492) 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91a3e00700 1 -- 172.21.15.50:0/1584436784 learned_addr learned my addr 172.21.15.50:0/1584436784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91a9000700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.977+0000 7f91a3400700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 0x55af4c4f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 -- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 msgr2=0x55af4c4f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 0x55af4c4f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 -- 172.21.15.50:0/1584436784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 msgr2=0x55af4c4f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 -- 172.21.15.50:0/1584436784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4c3d0420 con 0x55af4c4ee830 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a3400700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 0x55af4c4f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91a9000700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4f29e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a6150 tx=0x7f919c0052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7545965f0211d4ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f918e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c0a89f0 con 0x55af4c4ee830 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9194004ce0 con 0x55af4c4ee830 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.978+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f919c0b1ac0 con 0x55af4c4ee830 2024-05-10T21:16:38.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f919c0a95e0 con 0x55af4c4ee830 2024-05-10T21:16:38.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 10 client.26602 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:38.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af4c4ffe70 con 0x55af4c4ee830 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004ce0 con 0x55af4c4ee830 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f919c0a4850 con 0x55af4c4ee830 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 10 client.26602 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 20 client.26602 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c502400 0x55af4c504820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 4 client.26602 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.979+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af4c4b51c0 con 0x55af4c502400 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.980+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c502400 0x55af4c504820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.980+0000 7f91a3e00700 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c502400 0x55af4c504820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfd37b094b42927 server_cookie=98a5e03081ae07a4 in_seq=0 out_seq=0 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.980+0000 7f918e000700 10 client.26602 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.981+0000 7f918e000700 1 -- 172.21.15.50:0/1584436784 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af4c4b51c0 con 0x55af4c502400 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.981+0000 7f918e000700 10 client.26602 handle_command_reply: tid=0 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.981+0000 7f91b9b97b80 1 client.26602 shutdown 2024-05-10T21:16:38.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 msgr2=0x55af4c4f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4f29e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a6150 tx=0x7f919c0052b0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 shutdown_connections 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c502400 0x55af4c504820 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4ee830 0x55af4c4f29e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/1584436784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4e89f0 0x55af4c4f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f918e000700 0 client.26602 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 >> 172.21.15.50:0/1584436784 conn(0x55af4c3aff70 msgr2=0x55af4c4e74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 shutdown_connections 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 1 -- 172.21.15.50:0/1584436784 wait complete. 2024-05-10T21:16:38.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:38.982+0000 7f91b9b97b80 20 client.26602 trim_cache size 0 max 16384 2024-05-10T21:16:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.007+0000 7f91b9b97b80 1 Processor -- start 2024-05-10T21:16:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.007+0000 7f91b9b97b80 1 -- start start 2024-05-10T21:16:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4e89f0 0x55af4c4ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41b3730 con 0x55af4c36ae40 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41b0f60 con 0x55af4c4ee830 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41aedd0 con 0x55af4c4e89f0 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52502/0 (socket says 172.21.15.50:52502) 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a3e00700 1 -- 172.21.15.50:0/725639953 learned_addr learned my addr 172.21.15.50:0/725639953 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a3400700 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4e89f0 0x55af4c4ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a9000700 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.008+0000 7f91a3e00700 1 -- 172.21.15.50:0/725639953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4e89f0 msgr2=0x55af4c4ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f91a3e00700 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4e89f0 0x55af4c4ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f91a3e00700 1 -- 172.21.15.50:0/725639953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 msgr2=0x55af4c4fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:39.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f91a3e00700 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f91a3e00700 1 -- 172.21.15.50:0/725639953 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4c4ec9a0 con 0x55af4c36ae40 2024-05-10T21:16:39.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f91a3e00700 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f91940095c0 tx=0x7f9194011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795ee1bc22f27abb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f9198c00700 1 -- 172.21.15.50:0/725639953 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194031070 con 0x55af4c36ae40 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f9198c00700 1 -- 172.21.15.50:0/725639953 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004ae0 con 0x55af4c36ae40 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.009+0000 7f9198c00700 1 -- 172.21.15.50:0/725639953 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919402b940 con 0x55af4c36ae40 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 -- 172.21.15.50:0/725639953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 msgr2=0x55af4c4e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f91940095c0 tx=0x7f9194011720 comp rx=0 tx=0).stop 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 -- 172.21.15.50:0/725639953 shutdown_connections 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c36ae40 0x55af4c4e84b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/725639953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c4e89f0 0x55af4c4ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.010+0000 7f91b9b97b80 1 -- 172.21.15.50:0/725639953 >> 172.21.15.50:0/725639953 conn(0x55af4c3aff70 msgr2=0x55af4c4ebd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:39.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.011+0000 7f91b9b97b80 1 -- 172.21.15.50:0/725639953 shutdown_connections 2024-05-10T21:16:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.011+0000 7f91b9b97b80 1 -- 172.21.15.50:0/725639953 wait complete. 2024-05-10T21:16:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.011+0000 7f91b9b97b80 1 -- start start 2024-05-10T21:16:39.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.011+0000 7f91b9b97b80 1 Processor -- start 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 0x55af4c4fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41af420 con 0x55af4c4e89f0 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41b0890 con 0x55af4c4ee830 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f91a41afaf0 con 0x55af4c36ae40 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91b9b97b80 10 client.0 _ll_register_callbacks cb 0x55af4c3b1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91a3400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4fdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52506/0 (socket says 172.21.15.50:52506) 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91a9000700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91a3e00700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 0x55af4c4fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.012+0000 7f91a3400700 1 -- 172.21.15.50:0/3712029531 learned_addr learned my addr 172.21.15.50:0/3712029531 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 -- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 msgr2=0x55af4c4fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 -- 172.21.15.50:0/3712029531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 msgr2=0x55af4c4fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 -- 172.21.15.50:0/3712029531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4c56bb70 con 0x55af4c36ae40 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a9000700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f91a3e00700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 0x55af4c4fd750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9194004720 tx=0x7f9194011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef871f0961fa416 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.013+0000 7f918e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.014+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194031070 con 0x55af4c36ae40 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.014+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9194005270 con 0x55af4c36ae40 2024-05-10T21:16:39.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.014+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9194004ae0 con 0x55af4c36ae40 2024-05-10T21:16:39.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.018+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f919403c050 con 0x55af4c36ae40 2024-05-10T21:16:39.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.018+0000 7f91b9b97b80 10 client.26608 fetch_fsmap learned FSMap version 33 2024-05-10T21:16:39.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.018+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55af4c56cde0 con 0x55af4c36ae40 2024-05-10T21:16:39.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004ae0 con 0x55af4c36ae40 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f919402bce0 con 0x55af4c36ae40 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 10 client.26608 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 20 client.26608 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c56d1e0 0x55af4c56f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 4 client.26608 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af4c418400 con 0x55af4c56d1e0 2024-05-10T21:16:39.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.019+0000 7f91a3400700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c56d1e0 0x55af4c56f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:39.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f91a3400700 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c56d1e0 0x55af4c56f600 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6347cb906fed35b4 server_cookie=fc16f8389fad853 in_seq=0 out_seq=0 2024-05-10T21:16:39.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f918e000700 10 client.26608 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f918e000700 1 -- 172.21.15.50:0/3712029531 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af4c418400 con 0x55af4c56d1e0 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f918e000700 10 client.26608 handle_command_reply: tid=0 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f91b9b97b80 1 client.26608 shutdown 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 msgr2=0x55af4c4fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.020+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 0x55af4c4fd750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9194004720 tx=0x7f9194011bb0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 shutdown_connections 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55af4c56d1e0 0x55af4c56f600 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55af4c4ee830 0x55af4c4fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af4c36ae40 0x55af4c4fd750 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 --2- 172.21.15.50:0/3712029531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55af4c4e89f0 0x55af4c4fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 >> 172.21.15.50:0/3712029531 conn(0x55af4c3aff70 msgr2=0x55af4c4dd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f918e000700 0 client.26608 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:39.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 shutdown_connections 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 1 -- 172.21.15.50:0/3712029531 wait complete. 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.021+0000 7f91b9b97b80 20 client.26608 trim_cache size 0 max 16384 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "f7e7dc4b-3d4a-4b30-b57a-7a8435e9b4d2": { 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "f7e7dc4b-3d4a-4b30-b57a-7a8435e9b4d2", 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:16:39.019 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:16:39.020 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:16:39.020 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:16:39.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 msgr2=0x7f91880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 0x7f91880b4d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9190009750 tx=0x7f9190005330 comp rx=0 tx=0).stop 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 msgr2=0x7f91a41a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f919c007ee0 tx=0x7f919c00d3b0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 shutdown_connections 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f91880b2880 0x7f91880b4d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91a410c5c0 0x7f91a41a1900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a4106580 0x7f91a419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 --2- 172.21.15.50:0/3551037406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91a4106ed0 0x7f91a419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:39.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 >> 172.21.15.50:0/3551037406 conn(0x7f91a4102080 msgr2=0x7f91a4104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:39.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 shutdown_connections 2024-05-10T21:16:39.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:16:39.022+0000 7f91ab000700 1 -- 172.21.15.50:0/3551037406 wait complete. 2024-05-10T21:16:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:39 smithi092 ceph-mon[34415]: pgmap v694: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-05-10T21:16:39.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:39 smithi142 ceph-mon[34846]: pgmap v694: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-05-10T21:16:39.381 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:39 smithi050 ceph-mon[29546]: pgmap v694: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-05-10T21:16:39.799 INFO:tasks.cephfs.filesystem:scrub status for tag:f7e7dc4b-3d4a-4b30-b57a-7a8435e9b4d2 - {'path': '/', 'tag': 'f7e7dc4b-3d4a-4b30-b57a-7a8435e9b4d2', 'options': 'recursive,force'} 2024-05-10T21:16:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:41 smithi092 ceph-mon[34415]: pgmap v695: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 725 op/s 2024-05-10T21:16:41.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:41 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:41.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:41 smithi142 ceph-mon[34846]: pgmap v695: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 725 op/s 2024-05-10T21:16:41.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:41 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:41 smithi050 ceph-mon[29546]: pgmap v695: 97 pgs: 97 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 725 op/s 2024-05-10T21:16:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:41 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:41.645 INFO:tasks.workunit.client.0.smithi050.stdout: 2024-05-10T21:16:41.645 INFO:tasks.workunit.client.0.smithi050.stdout:Final score for writes: 177 2024-05-10T21:16:41.645 INFO:tasks.workunit.client.0.smithi050.stdout:Final score for reads : 86099 2024-05-10T21:16:41.646 INFO:tasks.workunit.client.0.smithi050.stdout: 2024-05-10T21:16:41.648 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T21:16:41.648 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T21:16:43.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:43 smithi092 ceph-mon[34415]: pgmap v696: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 257 KiB/s wr, 869 op/s 2024-05-10T21:16:43.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:43 smithi142 ceph-mon[34846]: pgmap v696: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 257 KiB/s wr, 869 op/s 2024-05-10T21:16:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:43 smithi050 ceph-mon[29546]: pgmap v696: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 257 KiB/s wr, 869 op/s 2024-05-10T21:16:45.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:45 smithi092 ceph-mon[34415]: pgmap v697: 97 pgs: 97 active+clean; 610 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 469 KiB/s wr, 875 op/s 2024-05-10T21:16:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:45 smithi142 ceph-mon[34846]: pgmap v697: 97 pgs: 97 active+clean; 610 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 469 KiB/s wr, 875 op/s 2024-05-10T21:16:45.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:45 smithi050 ceph-mon[29546]: pgmap v697: 97 pgs: 97 active+clean; 610 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 469 KiB/s wr, 875 op/s 2024-05-10T21:16:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 -- 172.21.15.50:0/1890175028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b081099f0 msgr2=0x7f9b0810be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 --2- 172.21.15.50:0/1890175028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b081099f0 0x7f9b0810be10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009ca0 tx=0x7f9b0002a8c0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 -- 172.21.15.50:0/1890175028 shutdown_connections 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 --2- 172.21.15.50:0/1890175028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b081099f0 0x7f9b0810be10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 --2- 172.21.15.50:0/1890175028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b08104cf0 0x7f9b08105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 --2- 172.21.15.50:0/1890175028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08103ab0 0x7f9b081047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 -- 172.21.15.50:0/1890175028 >> 172.21.15.50:0/1890175028 conn(0x7f9b080fecc0 msgr2=0x7f9b08101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 -- 172.21.15.50:0/1890175028 shutdown_connections 2024-05-10T21:16:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.163+0000 7f9b07e00640 1 -- 172.21.15.50:0/1890175028 wait complete. 2024-05-10T21:16:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.164+0000 7f9b07e00640 1 Processor -- start 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.164+0000 7f9b07e00640 1 -- start start 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08104cf0 0x7f9b08075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 0x7f9b08075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0810f630 con 0x7f9b08104cf0 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0810f4b0 con 0x7f9b081099f0 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0810f7b0 con 0x7f9b08103ab0 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32920/0 (socket says 172.21.15.50:32920) 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 -- 172.21.15.50:0/3180192923 learned_addr learned my addr 172.21.15.50:0/3180192923 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06000640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08104cf0 0x7f9b08075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b07400640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 0x7f9b08075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 -- 172.21.15.50:0/3180192923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 msgr2=0x7f9b08075560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 0x7f9b08075560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 -- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08104cf0 msgr2=0x7f9b08075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.165+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08104cf0 0x7f9b08075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9b06a00640 1 -- 172.21.15.50:0/3180192923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b08075c90 con 0x7f9b08103ab0 2024-05-10T21:16:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9b07400640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 0x7f9b08075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e9f0 tx=0x7f9af800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9aff400640 1 -- 172.21.15.50:0/3180192923 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af800ccc0 con 0x7f9b08103ab0 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9aff400640 1 -- 172.21.15.50:0/3180192923 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af80035f0 con 0x7f9b08103ab0 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b081a4290 con 0x7f9b08103ab0 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9aff400640 1 -- 172.21.15.50:0/3180192923 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af800ce60 con 0x7f9b08103ab0 2024-05-10T21:16:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.166+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b081a4750 con 0x7f9b08103ab0 2024-05-10T21:16:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.170+0000 7f9b14a25740 1 Processor -- start 2024-05-10T21:16:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.170+0000 7f9b14a25740 1 -- start start 2024-05-10T21:16:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be38780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56484bd95550 con 0x56484ba662f0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56484bd3abc0 con 0x56484ba6e4d0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b14a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56484bd91510 con 0x56484ba8f1b0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9aff400640 1 -- 172.21.15.50:0/3180192923 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9af8003aa0 con 0x7f9b08103ab0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32932/0 (socket says 172.21.15.50:32932) 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b07400640 1 -- 172.21.15.50:0/3761548570 learned_addr learned my addr 172.21.15.50:0/3761548570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b06000640 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be38780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.171+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 -- 172.21.15.50:0/3761548570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 msgr2=0x56484be38780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be38780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 -- 172.21.15.50:0/3761548570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 msgr2=0x56484be31880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be31880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 -- 172.21.15.50:0/3761548570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56484baa7210 con 0x56484ba8f1b0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.172+0000 7f9b07400640 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031a90 tx=0x7f9b00009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931628455b059702 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9af3e00640 1 -- 172.21.15.50:0/3761548570 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00031650 con 0x56484ba8f1b0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9af3e00640 1 -- 172.21.15.50:0/3761548570 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00036040 con 0x56484ba8f1b0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9af3e00640 1 -- 172.21.15.50:0/3761548570 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004760 con 0x56484ba8f1b0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9aff400640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 0x7f9aec0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9aff400640 1 -- 172.21.15.50:0/3180192923 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9af80e24c0 con 0x7f9b08103ab0 2024-05-10T21:16:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.173+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 0x7f9aec0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 -- 172.21.15.50:0/3761548570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 msgr2=0x56484be31300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031a90 tx=0x7f9b00009420 comp rx=0 tx=0).stop 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b06a00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 0x7f9aec0b4d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56484be326e0 tx=0x7f9af80e8060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 -- 172.21.15.50:0/3761548570 shutdown_connections 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be38780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be31880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3761548570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba8f1b0 0x56484be31300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 -- 172.21.15.50:0/3761548570 >> 172.21.15.50:0/3761548570 conn(0x56484ba83300 msgr2=0x56484be2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 -- 172.21.15.50:0/3761548570 shutdown_connections 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.174+0000 7f9b14a25740 1 -- 172.21.15.50:0/3761548570 wait complete. 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.175+0000 7f9b14a25740 1 -- start start 2024-05-10T21:16:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.175+0000 7f9b14a25740 1 Processor -- start 2024-05-10T21:16:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.176+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.176+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32940/0 (socket says 172.21.15.50:32940) 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be3b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba8f1b0 0x56484be42260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56484be39170 con 0x56484ba8f1b0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56484bd4e660 con 0x56484ba6e4d0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56484be392f0 con 0x56484ba662f0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b06000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be3b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b07400640 1 -- 172.21.15.50:0/4075664642 learned_addr learned my addr 172.21.15.50:0/4075664642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b14a25740 10 client.0 _ll_register_callbacks cb 0x56484bd18400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b06000640 1 -- 172.21.15.50:0/4075664642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 msgr2=0x56484be3adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b06000640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.177+0000 7f9b06000640 1 -- 172.21.15.50:0/4075664642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba8f1b0 msgr2=0x56484be42260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b06000640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba8f1b0 0x56484be42260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b06000640 1 -- 172.21.15.50:0/4075664642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484bb73620 con 0x56484ba6e4d0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b07400640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b06000640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be3b300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b08076320 tx=0x7f9af4012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5ea954b803d260 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9af0c00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004030 con 0x56484ba6e4d0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9af40041d0 con 0x56484ba6e4d0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00036040 con 0x56484ba6e4d0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9af4023070 con 0x56484ba6e4d0 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b14a25740 10 client.26675 fetch_fsmap learned FSMap version 35 2024-05-10T21:16:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.178+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56484be45220 con 0x56484ba6e4d0 2024-05-10T21:16:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00036040 con 0x56484ba6e4d0 2024-05-10T21:16:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9af4003a20 con 0x56484ba6e4d0 2024-05-10T21:16:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 10 client.26675 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T21:16:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 20 client.26675 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba4dd80 0x56484be48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 4 client.26675 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.179+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56484bda3160 con 0x56484ba4dd80 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.180+0000 7f9b07400640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba4dd80 0x56484be48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.180+0000 7f9b07400640 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba4dd80 0x56484be48a20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2be0623882b987 server_cookie=9527e65688693d0b in_seq=0 out_seq=0 2024-05-10T21:16:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.180+0000 7f9af0c00640 10 client.26675 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9af0c00640 1 -- 172.21.15.50:0/4075664642 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56484bda3160 con 0x56484ba4dd80 2024-05-10T21:16:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9af0c00640 10 client.26675 handle_command_reply: tid=0 2024-05-10T21:16:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 client.26675 shutdown 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 msgr2=0x56484be3b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be3b300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b08076320 tx=0x7f9af4012b40 comp rx=0 tx=0).stop 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 shutdown_connections 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba4dd80 0x56484be48a20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba8f1b0 0x56484be42260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be3b300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 --2- 172.21.15.50:0/4075664642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9af0c00640 0 client.26675 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 >> 172.21.15.50:0/4075664642 conn(0x56484ba83300 msgr2=0x56484be2bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.185+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 shutdown_connections 2024-05-10T21:16:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.186+0000 7f9b14a25740 1 -- 172.21.15.50:0/4075664642 wait complete. 2024-05-10T21:16:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.186+0000 7f9b14a25740 20 client.26675 trim_cache size 0 max 16384 2024-05-10T21:16:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 Processor -- start 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 -- start start 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be42180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081afc20 con 0x56484ba662f0 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081ad430 con 0x56484ba6e4d0 2024-05-10T21:16:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.209+0000 7f9b14a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081ab280 con 0x56484ba83300 2024-05-10T21:16:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32942/0 (socket says 172.21.15.50:32942) 2024-05-10T21:16:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 -- 172.21.15.50:0/3625118827 learned_addr learned my addr 172.21.15.50:0/3625118827 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b06000640 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be42180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 -- 172.21.15.50:0/3625118827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 msgr2=0x56484be426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 -- 172.21.15.50:0/3625118827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 msgr2=0x56484be42180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be42180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 -- 172.21.15.50:0/3625118827 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56484be32480 con 0x56484ba83300 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.210+0000 7f9b07400640 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031a90 tx=0x7f9b0002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98eabc905283e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.211+0000 7f9af3400640 1 -- 172.21.15.50:0/3625118827 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004790 con 0x56484ba83300 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.211+0000 7f9af3400640 1 -- 172.21.15.50:0/3625118827 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00036040 con 0x56484ba83300 2024-05-10T21:16:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.211+0000 7f9af3400640 1 -- 172.21.15.50:0/3625118827 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000305c0 con 0x56484ba83300 2024-05-10T21:16:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- 172.21.15.50:0/3625118827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 msgr2=0x56484be4d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031a90 tx=0x7f9b0002a8c0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- 172.21.15.50:0/3625118827 shutdown_connections 2024-05-10T21:16:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba662f0 0x56484be42180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 --2- 172.21.15.50:0/3625118827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba83300 0x56484be4d210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- 172.21.15.50:0/3625118827 >> 172.21.15.50:0/3625118827 conn(0x56484be34b00 msgr2=0x56484be35250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- 172.21.15.50:0/3625118827 shutdown_connections 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- 172.21.15.50:0/3625118827 wait complete. 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.212+0000 7f9b14a25740 1 -- start start 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 Processor -- start 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba83300 0x56484be41330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081ab8f0 con 0x56484ba83300 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081acd60 con 0x56484ba6e4d0 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081abfc0 con 0x56484ba662f0 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b14a25740 10 client.0 _ll_register_callbacks cb 0x7f9af4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.213+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32946/0 (socket says 172.21.15.50:32946) 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 -- 172.21.15.50:0/2632328211 learned_addr learned my addr 172.21.15.50:0/2632328211 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba83300 0x56484be41330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b06000640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be40400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 -- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 msgr2=0x56484be40400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be40400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 -- 172.21.15.50:0/2632328211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba83300 msgr2=0x56484be41330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba83300 0x56484be41330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 -- 172.21.15.50:0/2632328211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484be3beb0 con 0x56484ba662f0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b06000640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be40400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9b07400640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031b50 tx=0x7f9b00031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5437bc3adda7de7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9af0c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.214+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0002f610 con 0x56484ba662f0 2024-05-10T21:16:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00036040 con 0x56484ba662f0 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f9b0002f7b0 con 0x56484ba662f0 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9b00067020 con 0x56484ba662f0 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 10 client.26626 fetch_fsmap learned FSMap version 35 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56484be429b0 con 0x56484ba662f0 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00036040 con 0x56484ba662f0 2024-05-10T21:16:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f9b00030a70 con 0x56484ba662f0 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 10 client.26626 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 10 client.26626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 20 client.26626 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.215+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba8f1b0 0x56484beb2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.216+0000 7f9b14a25740 4 client.26626 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.216+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56484bd9cfd0 con 0x56484ba8f1b0 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.216+0000 7f9b06000640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba8f1b0 0x56484beb2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.216+0000 7f9b06000640 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba8f1b0 0x56484beb2de0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33de808fda5793e server_cookie=929e712c75b8de4d in_seq=0 out_seq=0 2024-05-10T21:16:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.216+0000 7f9af0c00640 10 client.26626 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9af0c00640 1 -- 172.21.15.50:0/2632328211 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56484bd9cfd0 con 0x56484ba8f1b0 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9af0c00640 10 client.26626 handle_command_reply: tid=0 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 client.26626 shutdown 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 msgr2=0x56484be3fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031b50 tx=0x7f9b00031a20 comp rx=0 tx=0).stop 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 shutdown_connections 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x56484ba8f1b0 0x56484beb2de0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56484ba83300 0x56484be41330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56484ba6e4d0 0x56484be40400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9af0c00640 0 client.26626 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.235+0000 7f9b14a25740 1 --2- 172.21.15.50:0/2632328211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56484ba662f0 0x56484be3fec0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.236+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 >> 172.21.15.50:0/2632328211 conn(0x56484be34b00 msgr2=0x56484be2fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.236+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 shutdown_connections 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.236+0000 7f9b14a25740 1 -- 172.21.15.50:0/2632328211 wait complete. 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.236+0000 7f9b14a25740 20 client.26626 trim_cache size 0 max 16384 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:16:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:16:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 msgr2=0x7f9aec0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 0x7f9aec0b4d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56484be326e0 tx=0x7f9af80e8060 comp rx=0 tx=0).stop 2024-05-10T21:16:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 msgr2=0x7f9b08076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e9f0 tx=0x7f9af800eeb0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 shutdown_connections 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f9aec0b2860 0x7f9aec0b4d70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b081099f0 0x7f9b08075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b08104cf0 0x7f9b08075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 --2- 172.21.15.50:0/3180192923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b08103ab0 0x7f9b08076910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 >> 172.21.15.50:0/3180192923 conn(0x7f9b080fecc0 msgr2=0x7f9b081059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 shutdown_connections 2024-05-10T21:16:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:46.237+0000 7f9b07e00640 1 -- 172.21.15.50:0/3180192923 wait complete. 2024-05-10T21:16:46.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:46 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:46.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:46 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:46.411 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:46 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:47.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:47 smithi092 ceph-mon[34415]: pgmap v698: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 775 KiB/s wr, 875 op/s 2024-05-10T21:16:47.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:47 smithi142 ceph-mon[34846]: pgmap v698: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 775 KiB/s wr, 875 op/s 2024-05-10T21:16:47.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:47 smithi050 ceph-mon[29546]: pgmap v698: 97 pgs: 97 active+clean; 608 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 775 KiB/s wr, 875 op/s 2024-05-10T21:16:48.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:48 smithi092 ceph-mon[34415]: pgmap v699: 97 pgs: 97 active+clean; 601 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 786 KiB/s wr, 743 op/s 2024-05-10T21:16:48.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:48 smithi142 ceph-mon[34846]: pgmap v699: 97 pgs: 97 active+clean; 601 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 786 KiB/s wr, 743 op/s 2024-05-10T21:16:48.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:48 smithi050 ceph-mon[29546]: pgmap v699: 97 pgs: 97 active+clean; 601 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 786 KiB/s wr, 743 op/s 2024-05-10T21:16:49.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:49 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:16:49.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:49 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:16:49.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:49 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:16:49.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:49 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:16:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:49 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:16:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:49 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:16:50.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:50 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:16:50.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:50 smithi092 ceph-mon[34415]: pgmap v700: 97 pgs: 97 active+clean; 594 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 KiB/s wr, 479 op/s 2024-05-10T21:16:50.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:50 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:16:50.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:50 smithi142 ceph-mon[34846]: pgmap v700: 97 pgs: 97 active+clean; 594 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 KiB/s wr, 479 op/s 2024-05-10T21:16:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:50 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:16:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:50 smithi050 ceph-mon[29546]: pgmap v700: 97 pgs: 97 active+clean; 594 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 843 KiB/s wr, 479 op/s 2024-05-10T21:16:53.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:52 smithi092 ceph-mon[34415]: pgmap v701: 97 pgs: 97 active+clean; 587 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-05-10T21:16:53.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:52 smithi142 ceph-mon[34846]: pgmap v701: 97 pgs: 97 active+clean; 587 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-05-10T21:16:53.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:52 smithi050 ceph-mon[29546]: pgmap v701: 97 pgs: 97 active+clean; 587 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-05-10T21:16:54.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:53 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:54.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:53 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:54.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:53 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:55.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:54 smithi092 ceph-mon[34415]: pgmap v702: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-05-10T21:16:55.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:54 smithi142 ceph-mon[34846]: pgmap v702: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-05-10T21:16:55.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:54 smithi050 ceph-mon[29546]: pgmap v702: 97 pgs: 97 active+clean; 584 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-05-10T21:16:57.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:56 smithi092 ceph-mon[34415]: pgmap v703: 97 pgs: 97 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 378 op/s 2024-05-10T21:16:57.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:56 smithi142 ceph-mon[34846]: pgmap v703: 97 pgs: 97 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 378 op/s 2024-05-10T21:16:57.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:56 smithi050 ceph-mon[29546]: pgmap v703: 97 pgs: 97 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 378 op/s 2024-05-10T21:16:59.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:59 smithi092 ceph-mon[34415]: pgmap v704: 97 pgs: 97 active+clean; 589 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-10T21:16:59.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:16:59 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:59.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:59 smithi142 ceph-mon[34846]: pgmap v704: 97 pgs: 97 active+clean; 589 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-10T21:16:59.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:16:59 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:59.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:59 smithi050 ceph-mon[29546]: pgmap v704: 97 pgs: 97 active+clean; 589 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 467 op/s 2024-05-10T21:16:59.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:16:59 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:16:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 -- 172.21.15.50:0/3639493543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0078b50 msgr2=0x7f19b00798a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 --2- 172.21.15.50:0/3639493543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0078b50 0x7f19b00798a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009900 tx=0x7f19a002a410 comp rx=0 tx=0).stop 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 -- 172.21.15.50:0/3639493543 shutdown_connections 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 --2- 172.21.15.50:0/3639493543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b007aae0 0x7f19b007aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 --2- 172.21.15.50:0/3639493543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b0079de0 0x7f19b007a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 --2- 172.21.15.50:0/3639493543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0078b50 0x7f19b00798a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.478+0000 7f19b6000640 1 -- 172.21.15.50:0/3639493543 >> 172.21.15.50:0/3639493543 conn(0x7f19b0101070 msgr2=0x7f19b0103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.479+0000 7f19b6000640 1 -- 172.21.15.50:0/3639493543 shutdown_connections 2024-05-10T21:16:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.479+0000 7f19b6000640 1 -- 172.21.15.50:0/3639493543 wait complete. 2024-05-10T21:16:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 Processor -- start 2024-05-10T21:16:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 -- start start 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b0078b50 0x7f19b01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b007aae0 0x7f19b01a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115d90 con 0x7f19b0078b50 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115c10 con 0x7f19b007aae0 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.480+0000 7f19b6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19b0115f10 con 0x7f19b0079de0 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36054/0 (socket says 172.21.15.50:36054) 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19af400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b0078b50 0x7f19b01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 -- 172.21.15.50:0/253390518 learned_addr learned my addr 172.21.15.50:0/253390518 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19afe00640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b007aae0 0x7f19b01a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 -- 172.21.15.50:0/253390518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b007aae0 msgr2=0x7f19b01a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b007aae0 0x7f19b01a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 -- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b0078b50 msgr2=0x7f19b01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b0078b50 0x7f19b01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.481+0000 7f19aea00640 1 -- 172.21.15.50:0/253390518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b01a8500 con 0x7f19b0079de0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19aea00640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f199c00e9f0 tx=0x7f199c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19a7e00640 1 -- 172.21.15.50:0/253390518 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c00cd80 con 0x7f19b0079de0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19a7e00640 1 -- 172.21.15.50:0/253390518 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c0108e0 con 0x7f19b0079de0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b01a87f0 con 0x7f19b0079de0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19a7e00640 1 -- 172.21.15.50:0/253390518 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c018980 con 0x7f19b0079de0 2024-05-10T21:16:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.482+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b01a8d90 con 0x7f19b0079de0 2024-05-10T21:16:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 Processor -- start 2024-05-10T21:16:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 -- start start 2024-05-10T21:16:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a402680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a409580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f55a366550 con 0x55f55a0372f0 2024-05-10T21:16:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f55a30bbc0 con 0x55f55a0601b0 2024-05-10T21:16:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.486+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f55a362510 con 0x55f55a03f4d0 2024-05-10T21:16:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:43016/0 (socket says 172.21.15.50:43016) 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 -- 172.21.15.50:0/972840962 learned_addr learned my addr 172.21.15.50:0/972840962 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19af400640 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a402680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 -- 172.21.15.50:0/972840962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 msgr2=0x55f55a409580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a409580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 -- 172.21.15.50:0/972840962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 msgr2=0x55f55a402680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a402680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.487+0000 7f19afe00640 1 -- 172.21.15.50:0/972840962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55a078210 con 0x55f55a0601b0 2024-05-10T21:16:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.488+0000 7f19a7e00640 1 -- 172.21.15.50:0/253390518 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f199c018b20 con 0x7f19b0079de0 2024-05-10T21:16:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.488+0000 7f19afe00640 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009fd0 tx=0x7f19a80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00006c5237c29ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.488+0000 7f199be00640 1 -- 172.21.15.50:0/972840962 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8004d20 con 0x55f55a0601b0 2024-05-10T21:16:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.488+0000 7f199be00640 1 -- 172.21.15.50:0/972840962 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8004050 con 0x55f55a0601b0 2024-05-10T21:16:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.488+0000 7f199be00640 1 -- 172.21.15.50:0/972840962 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8005220 con 0x55f55a0601b0 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.489+0000 7f19bcf8a740 1 -- 172.21.15.50:0/972840962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 msgr2=0x55f55a402100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.489+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009fd0 tx=0x7f19a80118d0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.489+0000 7f19a7e00640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 0x7f19940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.489+0000 7f19bcf8a740 1 -- 172.21.15.50:0/972840962 shutdown_connections 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a409580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a402680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/972840962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a402100 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 -- 172.21.15.50:0/972840962 >> 172.21.15.50:0/972840962 conn(0x55f55a054300 msgr2=0x55f55a3fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19a7e00640 1 -- 172.21.15.50:0/253390518 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f199c0e24c0 con 0x7f19b0079de0 2024-05-10T21:16:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19af400640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 0x7f19940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 -- 172.21.15.50:0/972840962 shutdown_connections 2024-05-10T21:16:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.490+0000 7f19bcf8a740 1 -- 172.21.15.50:0/972840962 wait complete. 2024-05-10T21:16:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 -- start start 2024-05-10T21:16:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19af400640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 0x7f19940b4d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009230 tx=0x7f19a0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 Processor -- start 2024-05-10T21:16:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.491+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f55a409fd0 con 0x55f55a03f4d0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f55a31f660 con 0x55f55a0372f0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f55a40a150 con 0x55f55a0601b0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19bcf8a740 10 client.0 _ll_register_callbacks cb 0x55f55a2e9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36064/0 (socket says 172.21.15.50:36064) 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 -- 172.21.15.50:0/2582255122 learned_addr learned my addr 172.21.15.50:0/2582255122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19af400640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 -- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 msgr2=0x55f55a40bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 -- 172.21.15.50:0/2582255122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 msgr2=0x55f55a40c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19afe00640 1 -- 172.21.15.50:0/2582255122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55a144620 con 0x55f55a0601b0 2024-05-10T21:16:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.492+0000 7f19af400640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:16:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.493+0000 7f19afe00640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009fa0 tx=0x7f19a8004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2a0b3ea7ebef89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.493+0000 7f1998c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.493+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8028020 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.493+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f19a8004c50 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.493+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a8004050 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f19a8004e50 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 10 client.26635 fetch_fsmap learned FSMap version 37 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f55a415f90 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8004050 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f19a800f570 con 0x55f55a0601b0 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 10 client.26635 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 20 client.26635 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f55a01ed80 0x55f55a419570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 4 client.26635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:16:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.494+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f55a374160 con 0x55f55a01ed80 2024-05-10T21:16:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.495+0000 7f19af400640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f55a01ed80 0x55f55a419570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.495+0000 7f1998c00640 10 client.26635 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.495+0000 7f19af400640 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f55a01ed80 0x55f55a419570 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7933699e57ea67e2 server_cookie=94736b13792a51f4 in_seq=0 out_seq=0 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.496+0000 7f1998c00640 1 -- 172.21.15.50:0/2582255122 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f55a374160 con 0x55f55a01ed80 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.496+0000 7f1998c00640 10 client.26635 handle_command_reply: tid=0 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 client.26635 shutdown 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 msgr2=0x55f55a413060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009fa0 tx=0x7f19a8004490 comp rx=0 tx=0).stop 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 shutdown_connections 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f55a01ed80 0x55f55a419570 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0601b0 0x55f55a413060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/2582255122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 >> 172.21.15.50:0/2582255122 conn(0x55f55a054300 msgr2=0x55f55a3fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.497+0000 7f1998c00640 0 client.26635 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.498+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 shutdown_connections 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.498+0000 7f19bcf8a740 1 -- 172.21.15.50:0/2582255122 wait complete. 2024-05-10T21:16:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.498+0000 7f19bcf8a740 20 client.26635 trim_cache size 0 max 16384 2024-05-10T21:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.517+0000 7f19bcf8a740 1 Processor -- start 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.517+0000 7f19bcf8a740 1 -- start start 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a412f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01b43a0 con 0x55f55a0372f0 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01b1bb0 con 0x55f55a0601b0 2024-05-10T21:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01afa00 con 0x55f55a03f4d0 2024-05-10T21:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19af400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36072/0 (socket says 172.21.15.50:36072) 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 -- 172.21.15.50:0/390345673 learned_addr learned my addr 172.21.15.50:0/390345673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19afe00640 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a412f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 -- 172.21.15.50:0/390345673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 msgr2=0x55f55a412f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a412f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 -- 172.21.15.50:0/390345673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 msgr2=0x55f55a40bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.518+0000 7f19aea00640 1 -- 172.21.15.50:0/390345673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55a3fdff0 con 0x55f55a03f4d0 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.519+0000 7f19aea00640 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f199c0a6e30 tx=0x7f199c018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=151ce777e4cda7ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.519+0000 7f199b400640 1 -- 172.21.15.50:0/390345673 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0b22c0 con 0x55f55a03f4d0 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.519+0000 7f199b400640 1 -- 172.21.15.50:0/390345673 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c0b2970 con 0x55f55a03f4d0 2024-05-10T21:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.519+0000 7f199b400640 1 -- 172.21.15.50:0/390345673 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0a8e20 con 0x55f55a03f4d0 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 -- 172.21.15.50:0/390345673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 msgr2=0x55f55a40c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f199c0a6e30 tx=0x7f199c018420 comp rx=0 tx=0).stop 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 -- 172.21.15.50:0/390345673 shutdown_connections 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a03f4d0 0x55f55a40c100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a412f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/390345673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a0372f0 0x55f55a40bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 -- 172.21.15.50:0/390345673 >> 172.21.15.50:0/390345673 conn(0x55f55a054300 msgr2=0x55f55a406040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.520+0000 7f19bcf8a740 1 -- 172.21.15.50:0/390345673 shutdown_connections 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.521+0000 7f19bcf8a740 1 -- 172.21.15.50:0/390345673 wait complete. 2024-05-10T21:16:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.521+0000 7f19bcf8a740 1 -- start start 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.521+0000 7f19bcf8a740 1 Processor -- start 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a47f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01b0070 con 0x55f55a03f4d0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01b14e0 con 0x55f55a0601b0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01b0740 con 0x55f55a0372f0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19bcf8a740 10 client.0 _ll_register_callbacks cb 0x7f199c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19afe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a47f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19aea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36078/0 (socket says 172.21.15.50:36078) 2024-05-10T21:16:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 -- 172.21.15.50:0/1605858457 learned_addr learned my addr 172.21.15.50:0/1605858457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:16:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 -- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 msgr2=0x55f55a47f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a47f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 -- 172.21.15.50:0/1605858457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 msgr2=0x55f55a40ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19af400640 1 -- 172.21.15.50:0/1605858457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55a47fe10 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19aea00640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.522+0000 7f19afe00640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a47f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f19af400640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19b0078420 tx=0x7f19a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551df645efb06024 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f1998c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0035040 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199c0b2970 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f19a00053c0 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f19a0060070 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f19bcf8a740 10 client.26641 fetch_fsmap learned FSMap version 37 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.523+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f55a481020 con 0x55f55a0372f0 2024-05-10T21:16:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c0b2970 con 0x55f55a0372f0 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f19a0004bf0 con 0x55f55a0372f0 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 10 client.26641 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 20 client.26641 populate_metadata read hostname 'smithi050' 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f559ff41c0 0x55f55a4839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 4 client.26641 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f55a36dfd0 con 0x55f559ff41c0 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.524+0000 7f19aea00640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f559ff41c0 0x55f55a4839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.525+0000 7f19aea00640 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f559ff41c0 0x55f55a4839c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef098e96fcf14c1 server_cookie=ff65d0436d04961f in_seq=0 out_seq=0 2024-05-10T21:16:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.525+0000 7f1998c00640 10 client.26641 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f1998c00640 1 -- 172.21.15.50:0/1605858457 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f55a36dfd0 con 0x55f559ff41c0 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f1998c00640 10 client.26641 handle_command_reply: tid=0 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 client.26641 shutdown 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 msgr2=0x55f55a40e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19b0078420 tx=0x7f19a0009260 comp rx=0 tx=0).stop 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 shutdown_connections 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55f559ff41c0 0x55f55a4839c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f55a0601b0 0x55f55a47f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55a03f4d0 0x55f55a40ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 --2- 172.21.15.50:0/1605858457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f55a0372f0 0x55f55a40e6c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f1998c00640 0 client.26641 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.600+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 >> 172.21.15.50:0/1605858457 conn(0x55f55a054300 msgr2=0x55f55a40bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 shutdown_connections 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19bcf8a740 1 -- 172.21.15.50:0/1605858457 wait complete. 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19bcf8a740 20 client.26641 trim_cache size 0 max 16384 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:16:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:16:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 msgr2=0x7f19940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 0x7f19940b4d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009230 tx=0x7f19a0031400 comp rx=0 tx=0).stop 2024-05-10T21:16:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 msgr2=0x7f19b01a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:16:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.601+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f199c00e9f0 tx=0x7f199c00eeb0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 shutdown_connections 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f19940b2860 0x7f19940b4d70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19b0079de0 0x7f19b01a0f50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b007aae0 0x7f19b01a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 --2- 172.21.15.50:0/253390518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19b0078b50 0x7f19b01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 >> 172.21.15.50:0/253390518 conn(0x7f19b0101070 msgr2=0x7f19b0102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 shutdown_connections 2024-05-10T21:16:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:16:59.602+0000 7f19b6000640 1 -- 172.21.15.50:0/253390518 wait complete. 2024-05-10T21:17:00.360 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:00 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:17:00.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:00 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:17:00.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:00 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:17:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:01 smithi092 ceph-mon[34415]: pgmap v705: 97 pgs: 97 active+clean; 591 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-05-10T21:17:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:01.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:01 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:01 smithi142 ceph-mon[34846]: pgmap v705: 97 pgs: 97 active+clean; 591 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-05-10T21:17:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:01 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:01 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:01 smithi050 ceph-mon[29546]: pgmap v705: 97 pgs: 97 active+clean; 591 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-05-10T21:17:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:01 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:02 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:02 smithi092 ceph-mon[34415]: pgmap v706: 97 pgs: 97 active+clean; 572 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-10T21:17:02.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:02 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:02 smithi142 ceph-mon[34846]: pgmap v706: 97 pgs: 97 active+clean; 572 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-10T21:17:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:02 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:02 smithi050 ceph-mon[29546]: pgmap v706: 97 pgs: 97 active+clean; 572 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-05-10T21:17:03.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:03 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.876 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:03 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:03 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:04.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:04 smithi092 ceph-mon[34415]: pgmap v707: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-10T21:17:04.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:04 smithi142 ceph-mon[34846]: pgmap v707: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-10T21:17:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:04 smithi050 ceph-mon[29546]: pgmap v707: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-10T21:17:05.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:17:05.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:17:05.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:05.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:17:05.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:17:05.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:17:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:17:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:17:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:06 smithi092 ceph-mon[34415]: pgmap v708: 97 pgs: 97 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-05-10T21:17:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:06 smithi142 ceph-mon[34846]: pgmap v708: 97 pgs: 97 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-05-10T21:17:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:06 smithi050 ceph-mon[29546]: pgmap v708: 97 pgs: 97 active+clean; 558 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-05-10T21:17:09.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:08 smithi092 ceph-mon[34415]: pgmap v709: 97 pgs: 97 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-05-10T21:17:09.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:08 smithi142 ceph-mon[34846]: pgmap v709: 97 pgs: 97 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-05-10T21:17:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:08 smithi050 ceph-mon[29546]: pgmap v709: 97 pgs: 97 active+clean; 548 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 419 op/s 2024-05-10T21:17:09.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:17:10.270 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:11.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:10 smithi092 ceph-mon[34415]: pgmap v710: 97 pgs: 97 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 322 op/s 2024-05-10T21:17:11.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:10 smithi142 ceph-mon[34846]: pgmap v710: 97 pgs: 97 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 322 op/s 2024-05-10T21:17:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:10 smithi050 ceph-mon[29546]: pgmap v710: 97 pgs: 97 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 322 op/s 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.571+0000 7fb350600700 1 -- 172.21.15.50:0/92849646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb348109270 msgr2=0x7fb348117450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.571+0000 7fb350600700 1 --2- 172.21.15.50:0/92849646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb348109270 0x7fb348117450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3340098f0 tx=0x7fb334009c00 comp rx=0 tx=0).stop 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 -- 172.21.15.50:0/92849646 shutdown_connections 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 --2- 172.21.15.50:0/92849646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb348109270 0x7fb348117450 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 --2- 172.21.15.50:0/92849646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3481030a0 0x7fb3481034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 --2- 172.21.15.50:0/92849646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb348114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 -- 172.21.15.50:0/92849646 >> 172.21.15.50:0/92849646 conn(0x7fb3480fedc0 msgr2=0x7fb3481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 -- 172.21.15.50:0/92849646 shutdown_connections 2024-05-10T21:17:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.572+0000 7fb350600700 1 -- 172.21.15.50:0/92849646 wait complete. 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.573+0000 7fb350600700 1 Processor -- start 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 -- start start 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3481030a0 0x7fb3481a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb348109270 0x7fb3481a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb34811aab0 con 0x7fb3481030a0 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb34811a930 con 0x7fb3481039f0 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb350600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb34811ac30 con 0x7fb348109270 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40802/0 (socket says 172.21.15.50:40802) 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb34d200700 1 -- 172.21.15.50:0/2368782414 learned_addr learned my addr 172.21.15.50:0/2368782414 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb34dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3481030a0 0x7fb3481a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.574+0000 7fb34e600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb348109270 0x7fb3481a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 -- 172.21.15.50:0/2368782414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb348109270 msgr2=0x7fb3481a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb348109270 0x7fb3481a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 -- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3481030a0 msgr2=0x7fb3481a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3481030a0 0x7fb3481a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 -- 172.21.15.50:0/2368782414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3481a6370 con 0x7fb3481039f0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.575+0000 7fb34d200700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ea30 tx=0x7fb33c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.576+0000 7fb346000700 1 -- 172.21.15.50:0/2368782414 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00cc90 con 0x7fb3481039f0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.576+0000 7fb346000700 1 -- 172.21.15.50:0/2368782414 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c00ce30 con 0x7fb3481039f0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.576+0000 7fb346000700 1 -- 172.21.15.50:0/2368782414 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c010430 con 0x7fb3481039f0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.576+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3481a6660 con 0x7fb3481039f0 2024-05-10T21:17:12.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.576+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34810c6f0 con 0x7fb3481039f0 2024-05-10T21:17:12.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.581+0000 7fb346000700 1 -- 172.21.15.50:0/2368782414 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb33c003f40 con 0x7fb3481039f0 2024-05-10T21:17:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 Processor -- start 2024-05-10T21:17:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb346000700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 0x7fb3300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 -- start start 2024-05-10T21:17:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb346000700 1 -- 172.21.15.50:0/2368782414 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb33c0e24e0 con 0x7fb3481039f0 2024-05-10T21:17:12.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d7d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563e25c99a20 con 0x563e25d7dbf0 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e25c37760 con 0x563e25d77c50 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563e25c959e0 con 0x563e25bf9e40 2024-05-10T21:17:12.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.584+0000 7fb34e600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 0x7fb3300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35986/0 (socket says 172.21.15.50:35986) 2024-05-10T21:17:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 -- 172.21.15.50:0/3248715150 learned_addr learned my addr 172.21.15.50:0/3248715150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34dc00700 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d7d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34e600700 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 -- 172.21.15.50:0/3248715150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 msgr2=0x563e25d7d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d7d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 -- 172.21.15.50:0/3248715150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 msgr2=0x563e25d78060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d78060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.585+0000 7fb34d200700 1 -- 172.21.15.50:0/3248715150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e25be68d0 con 0x563e25d7dbf0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.586+0000 7fb34d200700 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009230 tx=0x7fb33c003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84ec94ff7cde18d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.586+0000 7fb34e600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 0x7fb3300b4e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb334009680 tx=0x7fb33400ff80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.586+0000 7fb33aa00700 1 -- 172.21.15.50:0/3248715150 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0aa1d0 con 0x563e25d7dbf0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.586+0000 7fb33aa00700 1 -- 172.21.15.50:0/3248715150 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c0a9ad0 con 0x563e25d7dbf0 2024-05-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.586+0000 7fb33aa00700 1 -- 172.21.15.50:0/3248715150 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0e2110 con 0x563e25d7dbf0 2024-05-10T21:17:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3248715150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 msgr2=0x563e25d77710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009230 tx=0x7fb33c003940 comp rx=0 tx=0).stop 2024-05-10T21:17:12.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3248715150 shutdown_connections 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d78060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d7d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3248715150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d77710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3248715150 >> 172.21.15.50:0/3248715150 conn(0x563e25c38a80 msgr2=0x563e25d72790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3248715150 shutdown_connections 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.587+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3248715150 wait complete. 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.588+0000 7fb35f18bb80 1 -- start start 2024-05-10T21:17:12.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.588+0000 7fb35f18bb80 1 Processor -- start 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d77c50 0x563e25d818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d7dbf0 0x563e25d81e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563e25d846c0 con 0x563e25bf9e40 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563e25bffb20 con 0x563e25d7dbf0 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563e25d84840 con 0x563e25d77c50 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb34dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb34dc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35998/0 (socket says 172.21.15.50:35998) 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb34dc00700 1 -- 172.21.15.50:0/2843219295 learned_addr learned my addr 172.21.15.50:0/2843219295 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb34d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d77c50 0x563e25d818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb34e600700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d7dbf0 0x563e25d81e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.589+0000 7fb35f18bb80 10 client.0 _ll_register_callbacks cb 0x563e25c2d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 -- 172.21.15.50:0/2843219295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d77c50 msgr2=0x563e25d818d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d77c50 0x563e25d818d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 -- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d7dbf0 msgr2=0x563e25d81e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d7dbf0 0x563e25d81e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 -- 172.21.15.50:0/2843219295 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e25c5f420 con 0x563e25bf9e40 2024-05-10T21:17:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb34dc00700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb340007b00 tx=0x7fb340007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc6eaf7dbbfa284 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb32f400700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340003c00 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c0a9ad0 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.590+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb340003da0 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb340025050 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 10 client.16563 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563e25d8f320 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c0a9ad0 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb340004730 con 0x563e25bf9e40 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 20 client.16563 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.591+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25d91900 0x563e25d93dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.592+0000 7fb35f18bb80 4 client.16563 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:12.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.592+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e25d44070 con 0x563e25d91900 2024-05-10T21:17:12.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.592+0000 7fb34d200700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25d91900 0x563e25d93dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.592+0000 7fb32f400700 10 client.16563 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.593+0000 7fb34d200700 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25d91900 0x563e25d93dd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19dec698fa060f6a server_cookie=6a687287b4ce270f in_seq=0 out_seq=0 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb32f400700 1 -- 172.21.15.50:0/2843219295 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e25d44070 con 0x563e25d91900 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb32f400700 10 client.16563 handle_command_reply: tid=0 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 client.16563 shutdown 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 msgr2=0x563e25d81390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb340007b00 tx=0x7fb340007e10 comp rx=0 tx=0).stop 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 shutdown_connections 2024-05-10T21:17:12.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25d91900 0x563e25d93dd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d7dbf0 0x563e25d81e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d81390 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/2843219295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d77c50 0x563e25d818d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 >> 172.21.15.50:0/2843219295 conn(0x563e25c38a80 msgr2=0x563e25d76700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.594+0000 7fb32f400700 0 client.16563 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.595+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 shutdown_connections 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.595+0000 7fb35f18bb80 1 -- 172.21.15.50:0/2843219295 wait complete. 2024-05-10T21:17:12.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.595+0000 7fb35f18bb80 20 client.16563 trim_cache size 0 max 16384 2024-05-10T21:17:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 Processor -- start 2024-05-10T21:17:12.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 -- start start 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d7dbf0 0x563e25d81c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3481c0310 con 0x563e25bf9e40 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb348113a40 con 0x563e25d77c50 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3481118f0 con 0x563e25d7dbf0 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.621+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40836/0 (socket says 172.21.15.50:40836) 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 -- 172.21.15.50:0/3107814590 learned_addr learned my addr 172.21.15.50:0/3107814590 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34e600700 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d7dbf0 0x563e25d81c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34dc00700 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 -- 172.21.15.50:0/3107814590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d7dbf0 msgr2=0x563e25d81c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d7dbf0 0x563e25d81c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 -- 172.21.15.50:0/3107814590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 msgr2=0x563e25d77710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d77710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 -- 172.21.15.50:0/3107814590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e25d8f150 con 0x563e25d77c50 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb34d200700 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb33c020040 tx=0x7fb33c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e3e9cbfc8751f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.622+0000 7fb33a000700 1 -- 172.21.15.50:0/3107814590 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0aa1d0 con 0x563e25d77c50 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.623+0000 7fb33a000700 1 -- 172.21.15.50:0/3107814590 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c0c2040 con 0x563e25d77c50 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.623+0000 7fb33a000700 1 -- 172.21.15.50:0/3107814590 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0e2110 con 0x563e25d77c50 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.623+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3107814590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 msgr2=0x563e25d7bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.623+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb33c020040 tx=0x7fb33c0cc040 comp rx=0 tx=0).stop 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3107814590 shutdown_connections 2024-05-10T21:17:12.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25d7dbf0 0x563e25d81c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25bf9e40 0x563e25d77710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/3107814590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d7bb00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3107814590 >> 172.21.15.50:0/3107814590 conn(0x563e25c38a80 msgr2=0x563e25d71e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3107814590 shutdown_connections 2024-05-10T21:17:12.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 -- 172.21.15.50:0/3107814590 wait complete. 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.624+0000 7fb35f18bb80 1 -- start start 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 Processor -- start 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d89a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d86a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb348111f40 con 0x563e25d7dbf0 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb348113370 con 0x563e25d77c50 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3481125d0 con 0x563e25bf9e40 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb35f18bb80 10 client.0 _ll_register_callbacks cb 0x563e25c2d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.625+0000 7fb34e600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d86a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d89a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40838/0 (socket says 172.21.15.50:40838) 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 -- 172.21.15.50:0/1094550812 learned_addr learned my addr 172.21.15.50:0/1094550812 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 -- 172.21.15.50:0/1094550812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 msgr2=0x563e25d89a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d89a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 -- 172.21.15.50:0/1094550812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 msgr2=0x563e25d86a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d86a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 -- 172.21.15.50:0/1094550812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e25d86fa0 con 0x563e25d77c50 2024-05-10T21:17:12.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34e600700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb34d200700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0a9a60 tx=0x7fb33c009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a379a9b8832ad2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb32f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.626+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0a9ad0 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fb33c0a60c0 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c0c2040 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb33c0b2930 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 10 client.26717 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563e25dfc750 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c0c2040 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb33c004230 con 0x563e25d77c50 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 10 client.26717 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 20 client.26717 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:12.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25dfc970 0x563e25dfed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 4 client.26717 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:17:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.627+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e25ca7400 con 0x563e25dfc970 2024-05-10T21:17:12.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.629+0000 7fb34dc00700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25dfc970 0x563e25dfed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.629+0000 7fb34dc00700 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25dfc970 0x563e25dfed90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a41124f8369e4f server_cookie=108ad5e5d396d91b in_seq=0 out_seq=0 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.629+0000 7fb32f400700 10 client.26717 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb32f400700 1 -- 172.21.15.50:0/1094550812 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563e25ca7400 con 0x563e25dfc970 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb32f400700 10 client.26717 handle_command_reply: tid=0 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 client.26717 shutdown 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 msgr2=0x563e25d89fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0a9a60 tx=0x7fb33c009820 comp rx=0 tx=0).stop 2024-05-10T21:17:12.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 shutdown_connections 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563e25dfc970 0x563e25dfed90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e25d7dbf0 0x563e25d86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e25bf9e40 0x563e25d89a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 --2- 172.21.15.50:0/1094550812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e25d77c50 0x563e25d89fd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 >> 172.21.15.50:0/1094550812 conn(0x563e25c38a80 msgr2=0x563e25d7b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.630+0000 7fb32f400700 0 client.26717 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.631+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 shutdown_connections 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.631+0000 7fb35f18bb80 1 -- 172.21.15.50:0/1094550812 wait complete. 2024-05-10T21:17:12.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.631+0000 7fb35f18bb80 20 client.26717 trim_cache size 0 max 16384 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 msgr2=0x7fb3300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 0x7fb3300b4e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb334009680 tx=0x7fb33400ff80 comp rx=0 tx=0).stop 2024-05-10T21:17:12.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 msgr2=0x7fb3481a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ea30 tx=0x7fb33c00edf0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 shutdown_connections 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb3300b2900 0x7fb3300b4e10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb348109270 0x7fb3481a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3481030a0 0x7fb3481a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 --2- 172.21.15.50:0/2368782414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3481039f0 0x7fb3481a1a40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.632+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 >> 172.21.15.50:0/2368782414 conn(0x7fb3480fedc0 msgr2=0x7fb3481164c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.633+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 shutdown_connections 2024-05-10T21:17:12.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:12.633+0000 7fb350600700 1 -- 172.21.15.50:0/2368782414 wait complete. 2024-05-10T21:17:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 -- 172.21.15.50:0/1951143233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 msgr2=0x7fdbf81047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1951143233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf81047b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009ca0 tx=0x7fdbf002a8c0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 -- 172.21.15.50:0/1951143233 shutdown_connections 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1951143233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1951143233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1951143233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf81047b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 -- 172.21.15.50:0/1951143233 >> 172.21.15.50:0/1951143233 conn(0x7fdbf80fecc0 msgr2=0x7fdbf8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 -- 172.21.15.50:0/1951143233 shutdown_connections 2024-05-10T21:17:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.817+0000 7fdbfd400640 1 -- 172.21.15.50:0/1951143233 wait complete. 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.818+0000 7fdbfd400640 1 Processor -- start 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 -- start start 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf8195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810f600 con 0x7fdbf8103ab0 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810f480 con 0x7fdbf8104cf0 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfd400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810f780 con 0x7fdbf81099f0 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf8195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36038/0 (socket says 172.21.15.50:36038) 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbfca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf8195060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55990/0 (socket says 172.21.15.50:55990) 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40852/0 (socket says 172.21.15.50:40852) 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.819+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1376857602 learned_addr learned my addr 172.21.15.50:0/1376857602 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1376857602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 msgr2=0x7fdbf8195060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf8195060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1376857602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 msgr2=0x7fdbf8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1376857602 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf81957f0 con 0x7fdbf8103ab0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000f80 tx=0x7fdbf0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf4c00640 1 -- 172.21.15.50:0/1376857602 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002f380 con 0x7fdbf8103ab0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.820+0000 7fdbf4c00640 1 -- 172.21.15.50:0/1376857602 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf00317c0 con 0x7fdbf8103ab0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.821+0000 7fdbf4c00640 1 -- 172.21.15.50:0/1376857602 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002fb70 con 0x7fdbf8103ab0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.821+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf8195a80 con 0x7fdbf8103ab0 2024-05-10T21:17:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.821+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81a8aa0 con 0x7fdbf8103ab0 2024-05-10T21:17:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.824+0000 7fdc05a0e740 1 Processor -- start 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.824+0000 7fdc05a0e740 1 -- start start 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf4c00640 1 -- 172.21.15.50:0/1376857602 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdbf002fd10 con 0x7fdbf8103ab0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x561509697300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x561509697880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5615095fb550 con 0x5615092cc2f0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5615095a0bc0 con 0x5615092d44d0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5615095f7510 con 0x5615092f51b0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x561509697880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbfca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x561509697300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x561509697880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36044/0 (socket says 172.21.15.50:36044) 2024-05-10T21:17:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40868/0 (socket says 172.21.15.50:40868) 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbf7e00640 1 -- 172.21.15.50:0/3620615179 learned_addr learned my addr 172.21.15.50:0/3620615179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.825+0000 7fdbfca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x561509697300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56004/0 (socket says 172.21.15.50:56004) 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 -- 172.21.15.50:0/3620615179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 msgr2=0x561509697300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x561509697300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 -- 172.21.15.50:0/3620615179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 msgr2=0x561509697880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x561509697880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 -- 172.21.15.50:0/3620615179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150930d210 con 0x5615092d44d0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbf7400640 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800aba0 tx=0x7fdbe800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9a8fcf88478d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbed200640 1 -- 172.21.15.50:0/3620615179 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005220 con 0x5615092d44d0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbed200640 1 -- 172.21.15.50:0/3620615179 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8004c50 con 0x5615092d44d0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.826+0000 7fdbed200640 1 -- 172.21.15.50:0/3620615179 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8018840 con 0x5615092d44d0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdbf4c00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 0x7fdbdc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdbf4c00640 1 -- 172.21.15.50:0/1376857602 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdbf0103610 con 0x7fdbf8103ab0 2024-05-10T21:17:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdbfca00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 0x7fdbdc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 -- 172.21.15.50:0/3620615179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 msgr2=0x56150969e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800aba0 tx=0x7fdbe800f790 comp rx=0 tx=0).stop 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 -- 172.21.15.50:0/3620615179 shutdown_connections 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x56150969e780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x561509697880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/3620615179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x561509697300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.827+0000 7fdc05a0e740 1 -- 172.21.15.50:0/3620615179 >> 172.21.15.50:0/3620615179 conn(0x5615092e9300 msgr2=0x561509692c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.828+0000 7fdc05a0e740 1 -- 172.21.15.50:0/3620615179 shutdown_connections 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.828+0000 7fdbfca00640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 0x7fdbdc0b4d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5615096984d0 tx=0x7fdbe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.828+0000 7fdc05a0e740 1 -- 172.21.15.50:0/3620615179 wait complete. 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.828+0000 7fdc05a0e740 1 -- start start 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.828+0000 7fdc05a0e740 1 Processor -- start 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092d44d0 0x5615096a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092f51b0 0x5615096a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56150969f1d0 con 0x5615092d44d0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5615095b4660 con 0x5615092f51b0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56150969f350 con 0x5615092cc2f0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdc05a0e740 10 client.0 _ll_register_callbacks cb 0x56150957e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092d44d0 0x5615096a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56018/0 (socket says 172.21.15.50:56018) 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1825579321 learned_addr learned my addr 172.21.15.50:0/1825579321 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.829+0000 7fdbfca00640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092f51b0 0x5615096a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092f51b0 msgr2=0x5615096a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092f51b0 0x5615096a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1825579321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092d44d0 msgr2=0x5615096a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092d44d0 0x5615096a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1825579321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615093d9620 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00cab90 tx=0x7fdbf002f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41fef3a08a92e6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbe1600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00b2040 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdbf00cae60 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8004c50 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdbf00c6ac0 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdc05a0e740 10 client.26671 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.830+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56150969f1d0 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8004c50 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdbf00d35d0 con 0x5615092cc2f0 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 10 client.26671 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 20 client.26671 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092b3d80 0x5615096aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 4 client.26671 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561509609160 con 0x5615092b3d80 2024-05-10T21:17:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.831+0000 7fdbf7400640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092b3d80 0x5615096aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.832+0000 7fdbf7400640 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092b3d80 0x5615096aea10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d255ea1ac3ab0ca server_cookie=31ab8ba2a40b3ab8 in_seq=0 out_seq=0 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.832+0000 7fdbe1600640 10 client.26671 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdbe1600640 1 -- 172.21.15.50:0/1825579321 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561509609160 con 0x5615092b3d80 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdbe1600640 10 client.26671 handle_command_reply: tid=0 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 client.26671 shutdown 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 msgr2=0x5615096a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00cab90 tx=0x7fdbf002f6a0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 shutdown_connections 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092b3d80 0x5615096aea10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092f51b0 0x5615096a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092d44d0 0x5615096a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdbe1600640 0 client.26671 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1825579321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092cc2f0 0x5615096a0e20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 >> 172.21.15.50:0/1825579321 conn(0x5615092e9300 msgr2=0x561509695ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 shutdown_connections 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1825579321 wait complete. 2024-05-10T21:17:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.833+0000 7fdc05a0e740 20 client.26671 trim_cache size 0 max 16384 2024-05-10T21:17:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 Processor -- start 2024-05-10T21:17:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 -- start start 2024-05-10T21:17:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x5615096a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x5615096a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81b3ff0 con 0x5615092cc2f0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81b1800 con 0x5615092d44d0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.850+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81af650 con 0x5615092f51b0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x5615096a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36064/0 (socket says 172.21.15.50:36064) 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7e00640 1 -- 172.21.15.50:0/1429180592 learned_addr learned my addr 172.21.15.50:0/1429180592 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbfca00640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x5615096a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 -- 172.21.15.50:0/1429180592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 msgr2=0x5615096a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x5615096a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 -- 172.21.15.50:0/1429180592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 msgr2=0x56150969b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 -- 172.21.15.50:0/1429180592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150968e4a0 con 0x5615092d44d0 2024-05-10T21:17:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbfca00640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x5615096a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.851+0000 7fdbf7400640 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x5615096a8250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000c00 tx=0x7fdbe8012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f79e7917abcc67f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.852+0000 7fdbe3e00640 1 -- 172.21.15.50:0/1429180592 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe802f070 con 0x5615092d44d0 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.852+0000 7fdbe3e00640 1 -- 172.21.15.50:0/1429180592 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8004bf0 con 0x5615092d44d0 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.852+0000 7fdbe3e00640 1 -- 172.21.15.50:0/1429180592 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80056d0 con 0x5615092d44d0 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1429180592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 msgr2=0x5615096a8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x5615096a8250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000c00 tx=0x7fdbe8012040 comp rx=0 tx=0).stop 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1429180592 shutdown_connections 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092f51b0 0x5615096a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092d44d0 0x5615096a8250 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/1429180592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092cc2f0 0x56150969b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1429180592 >> 172.21.15.50:0/1429180592 conn(0x5615092e9300 msgr2=0x5615096a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1429180592 shutdown_connections 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- 172.21.15.50:0/1429180592 wait complete. 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 -- start start 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.853+0000 7fdc05a0e740 1 Processor -- start 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092d44d0 0x5615096a5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092f51b0 0x561509714db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81afcc0 con 0x5615092f51b0 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81b1130 con 0x5615092cc2f0 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81b0390 con 0x5615092d44d0 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdc05a0e740 10 client.0 _ll_register_callbacks cb 0x7fdbf0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:40908/0 (socket says 172.21.15.50:40908) 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 -- 172.21.15.50:0/867177682 learned_addr learned my addr 172.21.15.50:0/867177682 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbfca00640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092f51b0 0x561509714db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7400640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092d44d0 0x5615096a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 -- 172.21.15.50:0/867177682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092d44d0 msgr2=0x5615096a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092d44d0 0x5615096a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 -- 172.21.15.50:0/867177682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092f51b0 msgr2=0x561509714db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.854+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092f51b0 0x561509714db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbf7e00640 1 -- 172.21.15.50:0/867177682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615096a8400 con 0x5615092cc2f0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbf7e00640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006010 tx=0x7fdbf0106d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94f403f1ecce2b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbe1600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00b2040 con 0x5615092cc2f0 2024-05-10T21:17:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fdbf00cb070 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8004bf0 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdbf00ca370 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdc05a0e740 10 client.26732 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561509716260 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.855+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8004bf0 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdbf00c9ef0 con 0x5615092cc2f0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 10 client.26732 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 20 client.26732 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092891c0 0x561509718a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 4 client.26732 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561509602fd0 con 0x5615092891c0 2024-05-10T21:17:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdbf7400640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092891c0 0x561509718a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdbf7400640 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092891c0 0x561509718a90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a373eb13926e4d server_cookie=63a3a6a38b958436 in_seq=0 out_seq=0 2024-05-10T21:17:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:12.856+0000 7fdbe1600640 10 client.26732 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:13.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:12 smithi092 ceph-mon[34415]: pgmap v711: 97 pgs: 97 active+clean; 536 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 338 op/s 2024-05-10T21:17:13.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:12 smithi142 ceph-mon[34846]: pgmap v711: 97 pgs: 97 active+clean; 536 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 338 op/s 2024-05-10T21:17:13.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:12 smithi050 ceph-mon[29546]: pgmap v711: 97 pgs: 97 active+clean; 536 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 338 op/s 2024-05-10T21:17:13.489 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:17:13.489 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:17:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.535+0000 7fdbe1600640 1 -- 172.21.15.50:0/867177682 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561509602fd0 con 0x5615092891c0 2024-05-10T21:17:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.535+0000 7fdbe1600640 10 client.26732 handle_command_reply: tid=0 2024-05-10T21:17:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.535+0000 7fdc05a0e740 1 client.26732 shutdown 2024-05-10T21:17:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 msgr2=0x5615096a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006010 tx=0x7fdbf0106d50 comp rx=0 tx=0).stop 2024-05-10T21:17:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 shutdown_connections 2024-05-10T21:17:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5615092891c0 0x561509718a90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615092d44d0 0x5615096a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615092f51b0 0x561509714db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 --2- 172.21.15.50:0/867177682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5615092cc2f0 0x5615096a4f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 >> 172.21.15.50:0/867177682 conn(0x5615092e9300 msgr2=0x561509695a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.536+0000 7fdbe1600640 0 client.26732 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.537+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 shutdown_connections 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.537+0000 7fdc05a0e740 1 -- 172.21.15.50:0/867177682 wait complete. 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.537+0000 7fdc05a0e740 20 client.26732 trim_cache size 0 max 16384 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:17:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:17:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 msgr2=0x7fdbdc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 0x7fdbdc0b4d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5615096984d0 tx=0x7fdbe4005f90 comp rx=0 tx=0).stop 2024-05-10T21:17:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 msgr2=0x7fdbf8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000f80 tx=0x7fdbf0003ba0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 shutdown_connections 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdbdc0b2860 0x7fdbdc0b4d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf81099f0 0x7fdbf8195060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8104cf0 0x7fdbf8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 --2- 172.21.15.50:0/1376857602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbf8103ab0 0x7fdbf8198510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.538+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 >> 172.21.15.50:0/1376857602 conn(0x7fdbf80fecc0 msgr2=0x7fdbf80ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.539+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 shutdown_connections 2024-05-10T21:17:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:13.539+0000 7fdbfd400640 1 -- 172.21.15.50:0/1376857602 wait complete. 2024-05-10T21:17:13.967 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:15.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:14 smithi092 ceph-mon[34415]: pgmap v712: 97 pgs: 97 active+clean; 526 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-05-10T21:17:15.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:14 smithi142 ceph-mon[34846]: pgmap v712: 97 pgs: 97 active+clean; 526 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-05-10T21:17:15.385 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:14 smithi050 ceph-mon[29546]: pgmap v712: 97 pgs: 97 active+clean; 526 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 -- 172.21.15.50:0/3481071298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 msgr2=0x7fd440109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 --2- 172.21.15.50:0/3481071298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd440109140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0099c0 tx=0x7fd42c009cd0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 -- 172.21.15.50:0/3481071298 shutdown_connections 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 --2- 172.21.15.50:0/3481071298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd440104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 --2- 172.21.15.50:0/3481071298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 0x7fd440110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 --2- 172.21.15.50:0/3481071298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd440109140 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 -- 172.21.15.50:0/3481071298 >> 172.21.15.50:0/3481071298 conn(0x7fd440079d10 msgr2=0x7fd44007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:16.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 -- 172.21.15.50:0/3481071298 shutdown_connections 2024-05-10T21:17:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.349+0000 7fd446400700 1 -- 172.21.15.50:0/3481071298 wait complete. 2024-05-10T21:17:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 Processor -- start 2024-05-10T21:17:16.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 -- start start 2024-05-10T21:17:16.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:16.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd44019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 0x7fd4401a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401142e0 con 0x7fd440109680 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd440114160 con 0x7fd440104400 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.351+0000 7fd446400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd440114460 con 0x7fd440108d30 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44668/0 (socket says 172.21.15.50:44668) 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 -- 172.21.15.50:0/3525972001 learned_addr learned my addr 172.21.15.50:0/3525972001 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445a00700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 0x7fd4401a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd43fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd44019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 -- 172.21.15.50:0/3525972001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 msgr2=0x7fd44019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd44019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 -- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 msgr2=0x7fd4401a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:16.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 0x7fd4401a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd445000700 1 -- 172.21.15.50:0/3525972001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4401a1d80 con 0x7fd440104400 2024-05-10T21:17:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.352+0000 7fd43fe00700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd44019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd445000700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e9a0 tx=0x7fd43400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43400cbd0 con 0x7fd440104400 2024-05-10T21:17:16.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4401a2070 con 0x7fd440104400 2024-05-10T21:17:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43400cd70 con 0x7fd440104400 2024-05-10T21:17:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434018680 con 0x7fd440104400 2024-05-10T21:17:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.353+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a9990 con 0x7fd440104400 2024-05-10T21:17:16.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.354+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4401058b0 con 0x7fd440104400 2024-05-10T21:17:16.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.357+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd434010430 con 0x7fd440104400 2024-05-10T21:17:16.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.359+0000 7fd43d600700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 0x7fd4280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:16.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.359+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fd4340e26d0 con 0x7fd440104400 2024-05-10T21:17:16.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.359+0000 7fd43fe00700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 0x7fd4280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:16.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.360+0000 7fd43fe00700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 0x7fd4280b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd42c000c00 tx=0x7fd42c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:16.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.360+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4340aa3c0 con 0x7fd440104400 2024-05-10T21:17:16.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.673+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4400512c0 con 0x7fd440104400 2024-05-10T21:17:16.675 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:17:16.675 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:16:57.907713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25991":{"gid":25991,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1113302816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1113302816},{"type":"v1","addr":"172.21.15.92:6837","nonce":1113302816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:17:16.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.678+0000 7fd43d600700 1 -- 172.21.15.50:0/3525972001 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11861 (secure 0 0 0) 0x7fd4340a9cc0 con 0x7fd440104400 2024-05-10T21:17:16.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.683+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 msgr2=0x7fd4280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.683+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 0x7fd4280b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd42c000c00 tx=0x7fd42c01a040 comp rx=0 tx=0).stop 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.683+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 msgr2=0x7fd44019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.683+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e9a0 tx=0x7fd43400ecb0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 shutdown_connections 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd4280b2880 0x7fd4280b4d90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd440104400 0x7fd44019ceb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd440109680 0x7fd4401a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 --2- 172.21.15.50:0/3525972001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd440108d30 0x7fd44019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:16.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 >> 172.21.15.50:0/3525972001 conn(0x7fd440079d10 msgr2=0x7fd44010f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:16.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 shutdown_connections 2024-05-10T21:17:16.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:16.684+0000 7fd446400700 1 -- 172.21.15.50:0/3525972001 wait complete. 2024-05-10T21:17:16.684 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 37 2024-05-10T21:17:17.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:16 smithi092 ceph-mon[34415]: pgmap v713: 97 pgs: 97 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2024-05-10T21:17:17.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:16 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3525972001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:17:17.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:16 smithi142 ceph-mon[34846]: pgmap v713: 97 pgs: 97 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2024-05-10T21:17:17.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:16 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3525972001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:17:17.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:16 smithi050 ceph-mon[29546]: pgmap v713: 97 pgs: 97 active+clean; 513 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2024-05-10T21:17:17.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:16 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3525972001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:17:17.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:17:18.074 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:19.247 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:18 smithi050 ceph-mon[29546]: pgmap v714: 97 pgs: 97 active+clean; 501 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-05-10T21:17:19.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:18 smithi092 ceph-mon[34415]: pgmap v714: 97 pgs: 97 active+clean; 501 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-05-10T21:17:19.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:18 smithi142 ceph-mon[34846]: pgmap v714: 97 pgs: 97 active+clean; 501 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2024-05-10T21:17:20.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 -- 172.21.15.50:0/671690164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c108d30 msgr2=0x7f6a9c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 --2- 172.21.15.50:0/671690164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c108d30 0x7f6a9c109140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a880099c0 tx=0x7f6a88009cd0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 -- 172.21.15.50:0/671690164 shutdown_connections 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 --2- 172.21.15.50:0/671690164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c104400 0x7f6a9c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 --2- 172.21.15.50:0/671690164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c109680 0x7f6a9c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 --2- 172.21.15.50:0/671690164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c108d30 0x7f6a9c109140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 -- 172.21.15.50:0/671690164 >> 172.21.15.50:0/671690164 conn(0x7f6a9c079d10 msgr2=0x7f6a9c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 -- 172.21.15.50:0/671690164 shutdown_connections 2024-05-10T21:17:20.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.475+0000 7f6a9be00700 1 -- 172.21.15.50:0/671690164 wait complete. 2024-05-10T21:17:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.477+0000 7f6a9be00700 1 Processor -- start 2024-05-10T21:17:20.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.477+0000 7f6a9be00700 1 -- start start 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.477+0000 7f6a9be00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 0x7f6a9c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 0x7f6a9c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9be00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1142e0 con 0x7f6a9c104400 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9be00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c114160 con 0x7f6a9c109680 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c114460 con 0x7f6a9c108d30 2024-05-10T21:17:20.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37924/0 (socket says 172.21.15.50:37924) 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 0x7f6a9c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9aa00700 1 -- 172.21.15.50:0/2113138545 learned_addr learned my addr 172.21.15.50:0/2113138545 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.478+0000 7f6a9a000700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 0x7f6a9c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 -- 172.21.15.50:0/2113138545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 msgr2=0x7f6a9c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 0x7f6a9c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 -- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 msgr2=0x7f6a9c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 -- 172.21.15.50:0/2113138545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c1a90d0 con 0x7f6a9c109680 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9a000700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 0x7f6a9c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9b400700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 0x7f6a9c076540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400efc0 tx=0x7f6a9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a93400700 1 -- 172.21.15.50:0/2113138545 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a940092e0 con 0x7f6a9c109680 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a93400700 1 -- 172.21.15.50:0/2113138545 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a94004bb0 con 0x7f6a9c109680 2024-05-10T21:17:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c1a9360 con 0x7f6a9c109680 2024-05-10T21:17:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.479+0000 7f6a93400700 1 -- 172.21.15.50:0/2113138545 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004020 con 0x7f6a9c109680 2024-05-10T21:17:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.480+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c1a9820 con 0x7f6a9c109680 2024-05-10T21:17:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.483+0000 7f6a93400700 1 -- 172.21.15.50:0/2113138545 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a9401b030 con 0x7f6a9c109680 2024-05-10T21:17:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.484+0000 7f6ab0674b80 1 Processor -- start 2024-05-10T21:17:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 -- start start 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6a93400700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 0x7f6a800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 0x5607035f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560703511a20 con 0x5607035efc40 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5607034af760 con 0x5607035f5a80 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56070350d9e0 con 0x560703471e40 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6a93400700 1 -- 172.21.15.50:0/2113138545 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f6a940e1ca0 con 0x7f6a9c109680 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35104/0 (socket says 172.21.15.50:35104) 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.485+0000 7f6a9aa00700 1 -- 172.21.15.50:0/2348896105 learned_addr learned my addr 172.21.15.50:0/2348896105 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9a000700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 0x7f6a800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9b400700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 0x5607035f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9a000700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 -- 172.21.15.50:0/2348896105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 msgr2=0x5607035ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 -- 172.21.15.50:0/2348896105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 msgr2=0x5607035f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 0x5607035f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 -- 172.21.15.50:0/2348896105 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070345e8d0 con 0x560703471e40 2024-05-10T21:17:20.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9b400700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 0x5607035f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.486+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0117e0 tx=0x7f6a8c011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fab67159d74928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.487+0000 7f6a9a000700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 0x7f6a800b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.487+0000 7f6a87e00700 1 -- 172.21.15.50:0/2348896105 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c01e070 con 0x560703471e40 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.487+0000 7f6a87e00700 1 -- 172.21.15.50:0/2348896105 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c00f930 con 0x560703471e40 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.487+0000 7f6a87e00700 1 -- 172.21.15.50:0/2348896105 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c019990 con 0x560703471e40 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 -- 172.21.15.50:0/2348896105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 msgr2=0x5607035f5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0117e0 tx=0x7f6a8c011af0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 -- 172.21.15.50:0/2348896105 shutdown_connections 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607035efc40 0x5607035f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560703471e40 0x5607035f5540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/2348896105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 -- 172.21.15.50:0/2348896105 >> 172.21.15.50:0/2348896105 conn(0x5607034b6f70 msgr2=0x5607035ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 -- 172.21.15.50:0/2348896105 shutdown_connections 2024-05-10T21:17:20.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.488+0000 7f6ab0674b80 1 -- 172.21.15.50:0/2348896105 wait complete. 2024-05-10T21:17:20.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.489+0000 7f6ab0674b80 1 -- start start 2024-05-10T21:17:20.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.489+0000 7f6ab0674b80 1 Processor -- start 2024-05-10T21:17:20.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.489+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.489+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5607035fc490 con 0x560703471e40 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560703477b20 con 0x5607035f5a80 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607035fc610 con 0x5607035efc40 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6ab0674b80 10 client.0 _ll_register_callbacks cb 0x5607034b8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37940/0 (socket says 172.21.15.50:37940) 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 -- 172.21.15.50:0/4100673833 learned_addr learned my addr 172.21.15.50:0/4100673833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9b400700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9a000700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 -- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 msgr2=0x5607035f9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 -- 172.21.15.50:0/4100673833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 msgr2=0x5607035f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.490+0000 7f6a9aa00700 1 -- 172.21.15.50:0/4100673833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607034d7420 con 0x560703471e40 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6a9b400700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0194f0 tx=0x7f6a8c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c4042a7acdb858 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6a84c00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c027060 con 0x560703471e40 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c00f930 con 0x560703471e40 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.491+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f6a8c004030 con 0x560703471e40 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c01e040 con 0x560703471e40 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 10 client.16602 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:20.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560703606f20 con 0x560703471e40 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c00f930 con 0x560703471e40 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6a8c021030 con 0x560703471e40 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 10 client.16602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 20 client.16602 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703609560 0x56070360b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 4 client.16602 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.492+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607035bc1c0 con 0x560703609560 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.493+0000 7f6a9a000700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703609560 0x56070360b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.493+0000 7f6a84c00700 10 client.16602 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.493+0000 7f6a9a000700 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703609560 0x56070360b980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1f360f15af91291 server_cookie=52f39dd4243076f7 in_seq=0 out_seq=0 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.494+0000 7f6a84c00700 1 -- 172.21.15.50:0/4100673833 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607035bc1c0 con 0x560703609560 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.494+0000 7f6a84c00700 10 client.16602 handle_command_reply: tid=0 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.494+0000 7f6ab0674b80 1 client.16602 shutdown 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.494+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 msgr2=0x5607035f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.494+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0194f0 tx=0x7f6a8c00fd10 comp rx=0 tx=0).stop 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 shutdown_connections 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703609560 0x56070360b980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035f9110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/4100673833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 >> 172.21.15.50:0/4100673833 conn(0x5607034b6f70 msgr2=0x5607035ee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6a84c00700 0 client.16602 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:17:20.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 shutdown_connections 2024-05-10T21:17:20.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 1 -- 172.21.15.50:0/4100673833 wait complete. 2024-05-10T21:17:20.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.495+0000 7f6ab0674b80 20 client.16602 trim_cache size 0 max 16384 2024-05-10T21:17:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.512+0000 7f6ab0674b80 1 Processor -- start 2024-05-10T21:17:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.512+0000 7f6ab0674b80 1 -- start start 2024-05-10T21:17:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b3590 con 0x560703471e40 2024-05-10T21:17:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b0d60 con 0x5607035f5a80 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1aec10 con 0x5607035efc40 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44712/0 (socket says 172.21.15.50:44712) 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9b400700 1 -- 172.21.15.50:0/3245856299 learned_addr learned my addr 172.21.15.50:0/3245856299 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035ef700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37952/0 (socket says 172.21.15.50:37952) 2024-05-10T21:17:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9a000700 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.513+0000 7f6a9b400700 1 -- 172.21.15.50:0/3245856299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 msgr2=0x5607035f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a9b400700 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a9b400700 1 -- 172.21.15.50:0/3245856299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 msgr2=0x5607035ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a9b400700 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a9b400700 1 -- 172.21.15.50:0/3245856299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607035f7680 con 0x5607035f5a80 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a9b400700 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a940a6590 tx=0x7f6a940a9860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa20db5ffdf78f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a87400700 1 -- 172.21.15.50:0/3245856299 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004d60 con 0x5607035f5a80 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a87400700 1 -- 172.21.15.50:0/3245856299 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a940a8e30 con 0x5607035f5a80 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.514+0000 7f6a87400700 1 -- 172.21.15.50:0/3245856299 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9404e450 con 0x5607035f5a80 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 -- 172.21.15.50:0/3245856299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 msgr2=0x5607035ea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a940a6590 tx=0x7f6a940a9860 comp rx=0 tx=0).stop 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 -- 172.21.15.50:0/3245856299 shutdown_connections 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035f5a80 0x5607035ea4f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x5607035ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/3245856299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035efc40 0x5607035f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 -- 172.21.15.50:0/3245856299 >> 172.21.15.50:0/3245856299 conn(0x5607034b6f70 msgr2=0x5607035f42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 -- 172.21.15.50:0/3245856299 shutdown_connections 2024-05-10T21:17:20.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.515+0000 7f6ab0674b80 1 -- 172.21.15.50:0/3245856299 wait complete. 2024-05-10T21:17:20.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.516+0000 7f6ab0674b80 1 -- start start 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.516+0000 7f6ab0674b80 1 Processor -- start 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x560703611580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 0x560703601de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1af260 con 0x560703471e40 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1b0690 con 0x5607035efc40 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1af8f0 con 0x5607035f5a80 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6ab0674b80 10 client.0 _ll_register_callbacks cb 0x5607034b8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44722/0 (socket says 172.21.15.50:44722) 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9a000700 1 -- 172.21.15.50:0/725542697 learned_addr learned my addr 172.21.15.50:0/725542697 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x560703611580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9b400700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 0x560703601de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9a000700 1 -- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 msgr2=0x560703601de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.517+0000 7f6a9a000700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 0x560703601de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9a000700 1 -- 172.21.15.50:0/725542697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 msgr2=0x560703611580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9a000700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x560703611580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9a000700 1 -- 172.21.15.50:0/725542697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560703610080 con 0x5607035efc40 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x560703611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9b400700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 0x560703601de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a9a000700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c06c3c0 tx=0x7f6a88009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469fcac82ef958b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a84c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037020 con 0x5607035efc40 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f6a88033420 con 0x5607035efc40 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a940a8e30 con 0x5607035efc40 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a88033620 con 0x5607035efc40 2024-05-10T21:17:20.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6ab0674b80 10 client.26756 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:20.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.518+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560703672f30 con 0x5607035efc40 2024-05-10T21:17:20.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a940a8e30 con 0x5607035efc40 2024-05-10T21:17:20.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6a8800fc00 con 0x5607035efc40 2024-05-10T21:17:20.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 10 client.26756 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 10 client.26756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 20 client.26756 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703673680 0x560703675aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 4 client.26756 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56070351f400 con 0x560703673680 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.519+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703673680 0x560703675aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6a9aa00700 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703673680 0x560703675aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eaee50f45c54474b server_cookie=91345111be9e05b1 in_seq=0 out_seq=0 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6a84c00700 10 client.26756 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6a84c00700 1 -- 172.21.15.50:0/725542697 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56070351f400 con 0x560703673680 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6a84c00700 10 client.26756 handle_command_reply: tid=0 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6ab0674b80 1 client.26756 shutdown 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.520+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 msgr2=0x5607036018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c06c3c0 tx=0x7f6a88009cd0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 shutdown_connections 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x560703673680 0x560703675aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607035f5a80 0x560703601de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560703471e40 0x560703611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 --2- 172.21.15.50:0/725542697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607035efc40 0x5607036018a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 >> 172.21.15.50:0/725542697 conn(0x5607034b6f70 msgr2=0x5607035f3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6a84c00700 0 client.26756 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 shutdown_connections 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 1 -- 172.21.15.50:0/725542697 wait complete. 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.521+0000 7f6ab0674b80 20 client.26756 trim_cache size 0 max 16384 2024-05-10T21:17:20.520 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:17:20.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 msgr2=0x7f6a800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 0x7f6a800b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88009230 comp rx=0 tx=0).stop 2024-05-10T21:17:20.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 msgr2=0x7f6a9c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:20.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 0x7f6a9c076540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400efc0 tx=0x7f6a9400c3a0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 shutdown_connections 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6a800b2880 0x7f6a800b4d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c109680 0x7f6a9c076540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a9c104400 0x7f6a9c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 --2- 172.21.15.50:0/2113138545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c108d30 0x7f6a9c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 >> 172.21.15.50:0/2113138545 conn(0x7f6a9c079d10 msgr2=0x7f6a9c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 shutdown_connections 2024-05-10T21:17:20.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:20.522+0000 7f6a9be00700 1 -- 172.21.15.50:0/2113138545 wait complete. 2024-05-10T21:17:21.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:21 smithi092 ceph-mon[34415]: pgmap v715: 97 pgs: 97 active+clean; 495 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-10T21:17:21.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:21 smithi142 ceph-mon[34846]: pgmap v715: 97 pgs: 97 active+clean; 495 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-10T21:17:21.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:21 smithi050 ceph-mon[29546]: pgmap v715: 97 pgs: 97 active+clean; 495 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-10T21:17:21.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:17:22.025 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:23.191 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:23 smithi050 ceph-mon[29546]: pgmap v716: 97 pgs: 97 active+clean; 488 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 358 op/s 2024-05-10T21:17:23.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:23 smithi092 ceph-mon[34415]: pgmap v716: 97 pgs: 97 active+clean; 488 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 358 op/s 2024-05-10T21:17:23.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:23 smithi142 ceph-mon[34846]: pgmap v716: 97 pgs: 97 active+clean; 488 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 358 op/s 2024-05-10T21:17:24.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 -- 172.21.15.50:0/1950940221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c106580 msgr2=0x7f768c106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 --2- 172.21.15.50:0/1950940221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c106580 0x7f768c106990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7678009990 tx=0x7f7678009ca0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 -- 172.21.15.50:0/1950940221 shutdown_connections 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 --2- 172.21.15.50:0/1950940221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106ed0 0x7f768c113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 --2- 172.21.15.50:0/1950940221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c10c5c0 0x7f768c10c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 --2- 172.21.15.50:0/1950940221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c106580 0x7f768c106990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 -- 172.21.15.50:0/1950940221 >> 172.21.15.50:0/1950940221 conn(0x7f768c102080 msgr2=0x7f768c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 -- 172.21.15.50:0/1950940221 shutdown_connections 2024-05-10T21:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.129+0000 7f7694400700 1 -- 172.21.15.50:0/1950940221 wait complete. 2024-05-10T21:17:24.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.130+0000 7f7694400700 1 Processor -- start 2024-05-10T21:17:24.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.130+0000 7f7694400700 1 -- start start 2024-05-10T21:17:24.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c106ed0 0x7f768c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c10c5c0 0x7f768c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1166f0 con 0x7f768c10c5c0 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f768c116570 con 0x7f768c106580 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7694400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f768c116870 con 0x7f768c106ed0 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7691a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7691a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41584/0 (socket says 172.21.15.50:41584) 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7691a00700 1 -- 172.21.15.50:0/1585535278 learned_addr learned my addr 172.21.15.50:0/1585535278 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7692400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c10c5c0 0x7f768c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.131+0000 7f7691000700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c106ed0 0x7f768c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 -- 172.21.15.50:0/1585535278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c106ed0 msgr2=0x7f768c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c106ed0 0x7f768c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 -- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c10c5c0 msgr2=0x7f768c1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c10c5c0 0x7f768c1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 -- 172.21.15.50:0/1585535278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c1a1fd0 con 0x7f768c106580 2024-05-10T21:17:24.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.132+0000 7f7691a00700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f768000e970 tx=0x7f768000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.133+0000 7f768a000700 1 -- 172.21.15.50:0/1585535278 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768000cbd0 con 0x7f768c106580 2024-05-10T21:17:24.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.133+0000 7f768a000700 1 -- 172.21.15.50:0/1585535278 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f768000cd70 con 0x7f768c106580 2024-05-10T21:17:24.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.133+0000 7f768a000700 1 -- 172.21.15.50:0/1585535278 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680010430 con 0x7f768c106580 2024-05-10T21:17:24.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.133+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c1a22c0 con 0x7f768c106580 2024-05-10T21:17:24.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.133+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c1a9be0 con 0x7f768c106580 2024-05-10T21:17:24.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.137+0000 7f768a000700 1 -- 172.21.15.50:0/1585535278 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7680004750 con 0x7f768c106580 2024-05-10T21:17:24.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.138+0000 7f76a300fb80 1 Processor -- start 2024-05-10T21:17:24.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.138+0000 7f76a300fb80 1 -- start start 2024-05-10T21:17:24.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f768a000700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 0x7f76740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb869f0 0x558d4cb86e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558d4caa8a20 con 0x558d4cb869f0 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558d4ca46760 con 0x558d4cb8c830 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f76a300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d4caa49e0 con 0x558d4ca08e40 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f768a000700 1 -- 172.21.15.50:0/1585535278 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f76800e2350 con 0x7f768c106580 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f7692400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 0x7f76740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f7691a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f7691a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41604/0 (socket says 172.21.15.50:41604) 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.139+0000 7f7691a00700 1 -- 172.21.15.50:0/4257162818 learned_addr learned my addr 172.21.15.50:0/4257162818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:24.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691000700 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7692400700 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb869f0 0x558d4cb86e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 -- 172.21.15.50:0/4257162818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 msgr2=0x558d4cb864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 -- 172.21.15.50:0/4257162818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb869f0 msgr2=0x558d4cb86e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb869f0 0x558d4cb86e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 -- 172.21.15.50:0/4257162818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4c9f58d0 con 0x558d4ca08e40 2024-05-10T21:17:24.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7691a00700 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76800a6c10 tx=0x7f76800a9890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1543706bac039 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.140+0000 7f7692400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 0x7f76740b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7684015a60 tx=0x7f7684015420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.141+0000 7f767ea00700 1 -- 172.21.15.50:0/4257162818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768004ae20 con 0x558d4ca08e40 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.141+0000 7f767ea00700 1 -- 172.21.15.50:0/4257162818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76800b2140 con 0x558d4ca08e40 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.141+0000 7f767ea00700 1 -- 172.21.15.50:0/4257162818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800add50 con 0x558d4ca08e40 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 -- 172.21.15.50:0/4257162818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 msgr2=0x558d4cb8c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76800a6c10 tx=0x7f76800a9890 comp rx=0 tx=0).stop 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 -- 172.21.15.50:0/4257162818 shutdown_connections 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb869f0 0x558d4cb86e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cb8c2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 --2- 172.21.15.50:0/4257162818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 -- 172.21.15.50:0/4257162818 >> 172.21.15.50:0/4257162818 conn(0x558d4ca4df70 msgr2=0x558d4cb812d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 -- 172.21.15.50:0/4257162818 shutdown_connections 2024-05-10T21:17:24.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.142+0000 7f76a300fb80 1 -- 172.21.15.50:0/4257162818 wait complete. 2024-05-10T21:17:24.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 -- start start 2024-05-10T21:17:24.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 Processor -- start 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cb90a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558d4cb93310 con 0x558d4cb8c830 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558d4ca0eb20 con 0x558d4ca08e40 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.143+0000 7f76a300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d4cb93490 con 0x558d4cb869f0 2024-05-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f76a300fb80 10 client.0 _ll_register_callbacks cb 0x558d4ca4fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7692400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cb90a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41588/0 (socket says 172.21.15.50:41588) 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 -- 172.21.15.50:0/3525997307 learned_addr learned my addr 172.21.15.50:0/3525997307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691000700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 -- 172.21.15.50:0/3525997307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 msgr2=0x558d4cb904f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb904f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 -- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 msgr2=0x558d4cb90a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cb90a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.144+0000 7f7691a00700 1 -- 172.21.15.50:0/3525997307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4ca6e420 con 0x558d4ca08e40 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f7691a00700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f768000c5b0 tx=0x7f7680005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b508bce2cd318981 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f7673400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800aa0a0 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d4cb531c0 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f76800a9e10 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76800b2800 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f76a300fb80 10 client.26768 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.145+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558d4cb9de10 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d4cb531c0 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f76800c6070 con 0x558d4ca08e40 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 10 client.26768 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 10 client.26768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 20 client.26768 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cba02e0 0x558d4cba2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 4 client.26768 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d4cab6400 con 0x558d4cba02e0 2024-05-10T21:17:24.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.146+0000 7f7691000700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cba02e0 0x558d4cba2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.147+0000 7f7691000700 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cba02e0 0x558d4cba2700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=204d0966c214cf8f server_cookie=a43362fec2858eaf in_seq=0 out_seq=0 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.147+0000 7f7673400700 10 client.26768 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f7673400700 1 -- 172.21.15.50:0/3525997307 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d4cab6400 con 0x558d4cba02e0 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f7673400700 10 client.26768 handle_command_reply: tid=0 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 client.26768 shutdown 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 msgr2=0x558d4cb8ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f768000c5b0 tx=0x7f7680005020 comp rx=0 tx=0).stop 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 shutdown_connections 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4ca08e40 0x558d4cb8ffb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cba02e0 0x558d4cba2700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cb90a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 --2- 172.21.15.50:0/3525997307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb904f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 >> 172.21.15.50:0/3525997307 conn(0x558d4ca4df70 msgr2=0x558d4cb854a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f7673400700 0 client.26768 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.148+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 shutdown_connections 2024-05-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.149+0000 7f76a300fb80 1 -- 172.21.15.50:0/3525997307 wait complete. 2024-05-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.149+0000 7f76a300fb80 20 client.26768 trim_cache size 0 max 16384 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.166+0000 7f76a300fb80 1 Processor -- start 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.166+0000 7f76a300fb80 1 -- start start 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.166+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 0x558d4cb864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb90870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f76a300fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1b3720 con 0x558d4ca08e40 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f76a300fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1b0ef0 con 0x558d4cb8c830 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f76a300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1aed80 con 0x558d4cb869f0 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7691a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 0x558d4cb864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7691a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 0x558d4cb864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45378/0 (socket says 172.21.15.50:45378) 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7691a00700 1 -- 172.21.15.50:0/2560086560 learned_addr learned my addr 172.21.15.50:0/2560086560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7692400700 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb90870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7691000700 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7692400700 1 -- 172.21.15.50:0/2560086560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 msgr2=0x558d4cb9c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7692400700 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb9c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7692400700 1 -- 172.21.15.50:0/2560086560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 msgr2=0x558d4cb864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.167+0000 7f7692400700 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 0x558d4cb864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.168+0000 7f7692400700 1 -- 172.21.15.50:0/2560086560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4cb8bd30 con 0x558d4cb8c830 2024-05-10T21:17:24.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.168+0000 7f7692400700 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb90870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7684017630 tx=0x7f768401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6dad9fc18e8718 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.168+0000 7f767e000700 1 -- 172.21.15.50:0/2560086560 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768402c070 con 0x558d4cb8c830 2024-05-10T21:17:24.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.168+0000 7f767e000700 1 -- 172.21.15.50:0/2560086560 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76840079b0 con 0x558d4cb8c830 2024-05-10T21:17:24.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.168+0000 7f767e000700 1 -- 172.21.15.50:0/2560086560 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76840278c0 con 0x558d4cb8c830 2024-05-10T21:17:24.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 -- 172.21.15.50:0/2560086560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 msgr2=0x558d4cb90870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb90870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7684017630 tx=0x7f768401d000 comp rx=0 tx=0).stop 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 -- 172.21.15.50:0/2560086560 shutdown_connections 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb8c830 0x558d4cb90870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4ca08e40 0x558d4cb864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 --2- 172.21.15.50:0/2560086560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4cb869f0 0x558d4cb9c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 -- 172.21.15.50:0/2560086560 >> 172.21.15.50:0/2560086560 conn(0x558d4ca4df70 msgr2=0x558d4cb8b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 -- 172.21.15.50:0/2560086560 shutdown_connections 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.169+0000 7f76a300fb80 1 -- 172.21.15.50:0/2560086560 wait complete. 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.170+0000 7f76a300fb80 1 -- start start 2024-05-10T21:17:24.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.170+0000 7f76a300fb80 1 Processor -- start 2024-05-10T21:17:24.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cba26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb869f0 0x558d4cba2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1af3d0 con 0x558d4cb8c830 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1b0820 con 0x558d4cb869f0 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1afa80 con 0x558d4ca08e40 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f76a300fb80 10 client.0 _ll_register_callbacks cb 0x558d4ca4fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f7692400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f7692400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45384/0 (socket says 172.21.15.50:45384) 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f7692400700 1 -- 172.21.15.50:0/987672918 learned_addr learned my addr 172.21.15.50:0/987672918 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f7691a00700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cba26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.171+0000 7f7691000700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb869f0 0x558d4cba2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 -- 172.21.15.50:0/987672918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb869f0 msgr2=0x558d4cba2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb869f0 0x558d4cba2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 -- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 msgr2=0x558d4cc08820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 -- 172.21.15.50:0/987672918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4cba3490 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7692400700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7691a00700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cba26a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f76800a9930 tx=0x7f7680009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a09ec591afacac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7673400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800a5600 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76840079b0 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f76800a8e10 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.172+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76800a53e0 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 10 client.26719 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558d4cc09da0 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76840079b0 con 0x558d4ca08e40 2024-05-10T21:17:24.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f76800c6040 con 0x558d4ca08e40 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 10 client.26719 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 10 client.26719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 20 client.26719 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cc0a310 0x558d4cc0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 4 client.26719 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.173+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d4cab04a0 con 0x558d4cc0a310 2024-05-10T21:17:24.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.174+0000 7f7691000700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cc0a310 0x558d4cc0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:24.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.174+0000 7f7691000700 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cc0a310 0x558d4cc0c730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d61b3ac652f5fcfd server_cookie=120cf6ff5a247886 in_seq=0 out_seq=0 2024-05-10T21:17:24.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.174+0000 7f7673400700 10 client.26719 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:17:24.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.174+0000 7f7673400700 1 -- 172.21.15.50:0/987672918 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d4cab04a0 con 0x558d4cc0a310 2024-05-10T21:17:24.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.174+0000 7f7673400700 10 client.26719 handle_command_reply: tid=0 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 client.26719 shutdown 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 msgr2=0x558d4cba26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cba26a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f76800a9930 tx=0x7f7680009790 comp rx=0 tx=0).stop 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 shutdown_connections 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x558d4cc0a310 0x558d4cc0c730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558d4cb8c830 0x558d4cc08820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d4ca08e40 0x558d4cba26a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 --2- 172.21.15.50:0/987672918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558d4cb869f0 0x558d4cba2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f7673400700 0 client.26719 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 >> 172.21.15.50:0/987672918 conn(0x558d4ca4df70 msgr2=0x558d4cb8b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 shutdown_connections 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 1 -- 172.21.15.50:0/987672918 wait complete. 2024-05-10T21:17:24.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.175+0000 7f76a300fb80 20 client.26719 trim_cache size 0 max 16384 2024-05-10T21:17:24.175 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:17:24.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 msgr2=0x7f76740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 0x7f76740b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7684015a60 tx=0x7f7684015420 comp rx=0 tx=0).stop 2024-05-10T21:17:24.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 msgr2=0x7f768c19d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f768000e970 tx=0x7f768000ec80 comp rx=0 tx=0).stop 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 shutdown_connections 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f76740b2900 0x7f76740b4e10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c106580 0x7f768c19d100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768c10c5c0 0x7f768c1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 --2- 172.21.15.50:0/1585535278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c106ed0 0x7f768c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 >> 172.21.15.50:0/1585535278 conn(0x7f768c102080 msgr2=0x7f768c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 shutdown_connections 2024-05-10T21:17:24.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:24.176+0000 7f7694400700 1 -- 172.21.15.50:0/1585535278 wait complete. 2024-05-10T21:17:25.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:17:25.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:25 smithi092 ceph-mon[34415]: pgmap v717: 97 pgs: 97 active+clean; 483 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 318 op/s 2024-05-10T21:17:25.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:25 smithi142 ceph-mon[34846]: pgmap v717: 97 pgs: 97 active+clean; 483 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 318 op/s 2024-05-10T21:17:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:25 smithi050 ceph-mon[29546]: pgmap v717: 97 pgs: 97 active+clean; 483 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 318 op/s 2024-05-10T21:17:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.791+0000 7f90fe800640 1 -- 172.21.15.50:0/3502403120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 msgr2=0x7f90f80754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.791+0000 7f90fe800640 1 --2- 172.21.15.50:0/3502403120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 0x7f90f80754d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009860 tx=0x7f90e402a340 comp rx=0 tx=0).stop 2024-05-10T21:17:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 -- 172.21.15.50:0/3502403120 shutdown_connections 2024-05-10T21:17:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 --2- 172.21.15.50:0/3502403120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f8075a10 0x7f90f810fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 --2- 172.21.15.50:0/3502403120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f81103e0 0x7f90f8112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 --2- 172.21.15.50:0/3502403120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 0x7f90f80754d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 -- 172.21.15.50:0/3502403120 >> 172.21.15.50:0/3502403120 conn(0x7f90f80fee50 msgr2=0x7f90f8101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 -- 172.21.15.50:0/3502403120 shutdown_connections 2024-05-10T21:17:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.792+0000 7f90fe800640 1 -- 172.21.15.50:0/3502403120 wait complete. 2024-05-10T21:17:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 Processor -- start 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 -- start start 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 0x7f90f81a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f8075a10 0x7f90f81a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8115dc0 con 0x7f90f80750c0 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8115c40 con 0x7f90f8075a10 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.794+0000 7f90fe800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8115f40 con 0x7f90f81103e0 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41648/0 (socket says 172.21.15.50:41648) 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 -- 172.21.15.50:0/3506478048 learned_addr learned my addr 172.21.15.50:0/3506478048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f8075a10 0x7f90f81a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 -- 172.21.15.50:0/3506478048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f8075a10 msgr2=0x7f90f81a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f8075a10 0x7f90f81a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 -- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 msgr2=0x7f90f81a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 0x7f90f81a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.795+0000 7f90f7e00640 1 -- 172.21.15.50:0/3506478048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f81077c0 con 0x7f90f81103e0 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.796+0000 7f90f7e00640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009fa0 tx=0x7f90f000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.796+0000 7f90efe00640 1 -- 172.21.15.50:0/3506478048 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004b20 con 0x7f90f81103e0 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.797+0000 7f90efe00640 1 -- 172.21.15.50:0/3506478048 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f0003e20 con 0x7f90f81103e0 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.797+0000 7f90efe00640 1 -- 172.21.15.50:0/3506478048 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000b570 con 0x7f90f81103e0 2024-05-10T21:17:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.797+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f8107ab0 con 0x7f90f81103e0 2024-05-10T21:17:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.797+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f8107fb0 con 0x7f90f81103e0 2024-05-10T21:17:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.805+0000 7f90efe00640 1 -- 172.21.15.50:0/3506478048 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f90f001d020 con 0x7f90f81103e0 2024-05-10T21:17:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.806+0000 7f9105818740 1 Processor -- start 2024-05-10T21:17:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.806+0000 7f9105818740 1 -- start start 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562bb820a550 con 0x562bb7f041b0 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562bb81afbc0 con 0x562bb7ee34d0 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f9105818740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562bb8206510 con 0x562bb7edb2f0 2024-05-10T21:17:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f90f6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f90f6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45428/0 (socket says 172.21.15.50:45428) 2024-05-10T21:17:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.807+0000 7f90f6a00640 1 -- 172.21.15.50:0/3380286931 learned_addr learned my addr 172.21.15.50:0/3380286931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f7400640 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f6a00640 1 -- 172.21.15.50:0/3380286931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 msgr2=0x562bb82a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f6a00640 1 -- 172.21.15.50:0/3380286931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 msgr2=0x562bb82a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.808+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.809+0000 7f90f6a00640 1 -- 172.21.15.50:0/3380286931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb7f1c210 con 0x562bb7ee34d0 2024-05-10T21:17:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.809+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82ad580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005bc0 tx=0x7f90e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fab01a7a947d4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.809+0000 7f90efe00640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 0x7f90dc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.810+0000 7f90efe00640 1 -- 172.21.15.50:0/3506478048 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f90f00e22a0 con 0x7f90f81103e0 2024-05-10T21:17:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.810+0000 7f90ecc00640 1 -- 172.21.15.50:0/3380286931 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8005020 con 0x562bb7ee34d0 2024-05-10T21:17:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.810+0000 7f90ecc00640 1 -- 172.21.15.50:0/3380286931 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e80047a0 con 0x562bb7ee34d0 2024-05-10T21:17:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.810+0000 7f90ecc00640 1 -- 172.21.15.50:0/3380286931 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800f7c0 con 0x562bb7ee34d0 2024-05-10T21:17:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f90f7400640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 0x7f90dc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 -- 172.21.15.50:0/3380286931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 msgr2=0x562bb82ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82ad580 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005bc0 tx=0x7f90e8004730 comp rx=0 tx=0).stop 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 -- 172.21.15.50:0/3380286931 shutdown_connections 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82ad580 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 --2- 172.21.15.50:0/3380286931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.812+0000 7f9105818740 1 -- 172.21.15.50:0/3380286931 >> 172.21.15.50:0/3380286931 conn(0x562bb7ef8300 msgr2=0x562bb82a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.813+0000 7f9105818740 1 -- 172.21.15.50:0/3380286931 shutdown_connections 2024-05-10T21:17:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.813+0000 7f9105818740 1 -- 172.21.15.50:0/3380286931 wait complete. 2024-05-10T21:17:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.814+0000 7f90f7400640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 0x7f90dc0b4d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90e402a930 tx=0x7f90e40303b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.814+0000 7f9105818740 1 -- start start 2024-05-10T21:17:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.814+0000 7f9105818740 1 Processor -- start 2024-05-10T21:17:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562bb82adfd0 con 0x562bb7f041b0 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562bb81c3660 con 0x562bb7ee34d0 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562bb82ae150 con 0x562bb7edb2f0 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f9105818740 10 client.0 _ll_register_callbacks cb 0x562bb818d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45444/0 (socket says 172.21.15.50:45444) 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.816+0000 7f90f7e00640 1 -- 172.21.15.50:0/1117185201 learned_addr learned my addr 172.21.15.50:0/1117185201 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 -- 172.21.15.50:0/1117185201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 msgr2=0x562bb82afc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82afc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 -- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 msgr2=0x562bb82b01c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82b01c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 -- 172.21.15.50:0/1117185201 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb7fe8620 con 0x562bb7f041b0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90f7e00640 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90f00a9590 tx=0x7f90f00a9720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763e4c6f0f20a69e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.817+0000 7f90e1600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.818+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f001a070 con 0x562bb7f041b0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.818+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f90f00a5e80 con 0x562bb7f041b0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.818+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e80047a0 con 0x562bb7f041b0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.818+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90f00a93a0 con 0x562bb7f041b0 2024-05-10T21:17:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.819+0000 7f9105818740 10 client.16635 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.819+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562bb82ba110 con 0x562bb7f041b0 2024-05-10T21:17:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.820+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e80047a0 con 0x562bb7f041b0 2024-05-10T21:17:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.820+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f90f00bece0 con 0x562bb7f041b0 2024-05-10T21:17:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 10 client.16635 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 20 client.16635 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7ec2d80 0x562bb82bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 4 client.16635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.821+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb8218160 con 0x562bb7ec2d80 2024-05-10T21:17:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.822+0000 7f90f7400640 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7ec2d80 0x562bb82bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.824+0000 7f90f7400640 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7ec2d80 0x562bb82bd830 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4edc19d56f0548 server_cookie=95a382336ce6626c in_seq=0 out_seq=0 2024-05-10T21:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.825+0000 7f90e1600640 10 client.16635 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.826+0000 7f90e1600640 1 -- 172.21.15.50:0/1117185201 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562bb8218160 con 0x562bb7ec2d80 2024-05-10T21:17:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.826+0000 7f90e1600640 10 client.16635 handle_command_reply: tid=0 2024-05-10T21:17:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.826+0000 7f9105818740 1 client.16635 shutdown 2024-05-10T21:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 msgr2=0x562bb82b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90f00a9590 tx=0x7f90f00a9720 comp rx=0 tx=0).stop 2024-05-10T21:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 shutdown_connections 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7ec2d80 0x562bb82bd830 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7f041b0 0x562bb82b7120 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7ee34d0 0x562bb82b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 --2- 172.21.15.50:0/1117185201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7edb2f0 0x562bb82afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 >> 172.21.15.50:0/1117185201 conn(0x562bb7ef8300 msgr2=0x562bb82a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.827+0000 7f90e1600640 0 client.16635 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.828+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 shutdown_connections 2024-05-10T21:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.828+0000 7f9105818740 1 -- 172.21.15.50:0/1117185201 wait complete. 2024-05-10T21:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.829+0000 7f9105818740 20 client.16635 trim_cache size 0 max 16384 2024-05-10T21:17:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.881+0000 7f9105818740 1 Processor -- start 2024-05-10T21:17:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.881+0000 7f9105818740 1 -- start start 2024-05-10T21:17:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.881+0000 7f9105818740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 0x562bb82aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.881+0000 7f9105818740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 0x562bb82a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.882+0000 7f90f7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 0x562bb82aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.882+0000 7f90f7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 0x562bb82aa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45456/0 (socket says 172.21.15.50:45456) 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f9105818740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7f041b0 0x562bb82b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f9105818740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81bcb20 con 0x562bb7edb2f0 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f9105818740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81ba330 con 0x562bb7f041b0 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f9105818740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810eca0 con 0x562bb7ee34d0 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f90f7400640 1 -- 172.21.15.50:0/3189353238 learned_addr learned my addr 172.21.15.50:0/3189353238 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 0x562bb82a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.883+0000 7f90f7e00640 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7f041b0 0x562bb82b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.884+0000 7f90f6a00640 1 -- 172.21.15.50:0/3189353238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7f041b0 msgr2=0x562bb82b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.884+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7f041b0 0x562bb82b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.884+0000 7f90f6a00640 1 -- 172.21.15.50:0/3189353238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 msgr2=0x562bb82aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.884+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 0x562bb82aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.884+0000 7f90f6a00640 1 -- 172.21.15.50:0/3189353238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb82ad4b0 con 0x562bb7ee34d0 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.885+0000 7f90f6a00640 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 0x562bb82a1870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f90e8016000 tx=0x7f90e8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af4a13379355748 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.885+0000 7f90e3e00640 1 -- 172.21.15.50:0/3189353238 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8013170 con 0x562bb7ee34d0 2024-05-10T21:17:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.886+0000 7f90e3e00640 1 -- 172.21.15.50:0/3189353238 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e80133e0 con 0x562bb7ee34d0 2024-05-10T21:17:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.886+0000 7f90e3e00640 1 -- 172.21.15.50:0/3189353238 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800f580 con 0x562bb7ee34d0 2024-05-10T21:17:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.887+0000 7f9105818740 1 -- 172.21.15.50:0/3189353238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 msgr2=0x562bb82a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.887+0000 7f9105818740 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 0x562bb82a1870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f90e8016000 tx=0x7f90e8004600 comp rx=0 tx=0).stop 2024-05-10T21:17:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 -- 172.21.15.50:0/3189353238 shutdown_connections 2024-05-10T21:17:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7ee34d0 0x562bb82a1870 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7f041b0 0x562bb82b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 --2- 172.21.15.50:0/3189353238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7edb2f0 0x562bb82aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 -- 172.21.15.50:0/3189353238 >> 172.21.15.50:0/3189353238 conn(0x562bb7ef8300 msgr2=0x562bb82a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 -- 172.21.15.50:0/3189353238 shutdown_connections 2024-05-10T21:17:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.888+0000 7f9105818740 1 -- 172.21.15.50:0/3189353238 wait complete. 2024-05-10T21:17:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.889+0000 7f9105818740 1 -- start start 2024-05-10T21:17:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.890+0000 7f9105818740 1 Processor -- start 2024-05-10T21:17:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.892+0000 7f9105818740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7ee34d0 0x562bb82b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7f041b0 0x562bb82b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810f2f0 con 0x562bb7ee34d0 2024-05-10T21:17:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81b9c60 con 0x562bb7edb2f0 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810f9c0 con 0x562bb7f041b0 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.893+0000 7f9105818740 10 client.0 _ll_register_callbacks cb 0x7f90f0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.894+0000 7f90f7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.894+0000 7f90f7400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41654/0 (socket says 172.21.15.50:41654) 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.894+0000 7f90f7400640 1 -- 172.21.15.50:0/889676677 learned_addr learned my addr 172.21.15.50:0/889676677 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.894+0000 7f90f7e00640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7f041b0 0x562bb82b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.895+0000 7f90f7400640 1 -- 172.21.15.50:0/889676677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7f041b0 msgr2=0x562bb82b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.895+0000 7f90f7400640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7f041b0 0x562bb82b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.895+0000 7f90f7400640 1 -- 172.21.15.50:0/889676677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7ee34d0 msgr2=0x562bb82b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.895+0000 7f90f6a00640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7ee34d0 0x562bb82b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.896+0000 7f90f7400640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7ee34d0 0x562bb82b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.896+0000 7f90f7400640 1 -- 172.21.15.50:0/889676677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb82b6a80 con 0x562bb7edb2f0 2024-05-10T21:17:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.897+0000 7f90f7400640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f90e4035040 tx=0x7f90e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d0e185e9cbd862 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.897+0000 7f90e1600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.898+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4044040 con 0x562bb7edb2f0 2024-05-10T21:17:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.899+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7f90e4005040 con 0x562bb7edb2f0 2024-05-10T21:17:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.899+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e80133e0 con 0x562bb7edb2f0 2024-05-10T21:17:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.900+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90e405a050 con 0x562bb7edb2f0 2024-05-10T21:17:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.900+0000 7f9105818740 10 client.26789 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.900+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562bb8325be0 con 0x562bb7edb2f0 2024-05-10T21:17:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.901+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e80133e0 con 0x562bb7edb2f0 2024-05-10T21:17:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.901+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f90e40035f0 con 0x562bb7edb2f0 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 10 client.26789 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 20 client.26789 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7e981c0 0x562bb8328610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 4 client.26789 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562bb8211fd0 con 0x562bb7e981c0 2024-05-10T21:17:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.902+0000 7f90f6a00640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7e981c0 0x562bb8328610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.903+0000 7f90f6a00640 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7e981c0 0x562bb8328610 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d129e554e67a309c server_cookie=1978b01d9f029e74 in_seq=0 out_seq=0 2024-05-10T21:17:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:26.903+0000 7f90e1600640 10 client.26789 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.001+0000 7f90e1600640 1 -- 172.21.15.50:0/889676677 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562bb8211fd0 con 0x562bb7e981c0 2024-05-10T21:17:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.001+0000 7f90e1600640 10 client.26789 handle_command_reply: tid=0 2024-05-10T21:17:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.001+0000 7f9105818740 1 client.26789 shutdown 2024-05-10T21:17:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.001+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 msgr2=0x562bb82b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.001+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f90e4035040 tx=0x7f90e4005f90 comp rx=0 tx=0).stop 2024-05-10T21:17:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 shutdown_connections 2024-05-10T21:17:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x562bb7e981c0 0x562bb8328610 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562bb7f041b0 0x562bb82b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562bb7ee34d0 0x562bb82b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f90e1600640 0 client.26789 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 --2- 172.21.15.50:0/889676677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562bb7edb2f0 0x562bb82b3040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 >> 172.21.15.50:0/889676677 conn(0x562bb7ef8300 msgr2=0x562bb82c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 shutdown_connections 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 1 -- 172.21.15.50:0/889676677 wait complete. 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.002+0000 7f9105818740 20 client.26789 trim_cache size 0 max 16384 2024-05-10T21:17:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:17:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:17:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:17:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:17:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 msgr2=0x7f90dc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 0x7f90dc0b4d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90e402a930 tx=0x7f90e40303b0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 msgr2=0x7f90f81a7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009fa0 tx=0x7f90f000c800 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 shutdown_connections 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f90dc0b2860 0x7f90dc0b4d70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f81103e0 0x7f90f81a7e50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f8075a10 0x7f90f81a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 --2- 172.21.15.50:0/3506478048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90f80750c0 0x7f90f81a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.003+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 >> 172.21.15.50:0/3506478048 conn(0x7f90f80fee50 msgr2=0x7f90f8103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.004+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 shutdown_connections 2024-05-10T21:17:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:27.004+0000 7f90fe800640 1 -- 172.21.15.50:0/3506478048 wait complete. 2024-05-10T21:17:27.244 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:27 smithi050 ceph-mon[29546]: pgmap v718: 97 pgs: 97 active+clean; 481 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 954 KiB/s wr, 270 op/s 2024-05-10T21:17:27.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:27 smithi092 ceph-mon[34415]: pgmap v718: 97 pgs: 97 active+clean; 481 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 954 KiB/s wr, 270 op/s 2024-05-10T21:17:27.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:27 smithi142 ceph-mon[34846]: pgmap v718: 97 pgs: 97 active+clean; 481 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 954 KiB/s wr, 270 op/s 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.915+0000 7fad88000700 1 -- 172.21.15.50:0/2349319433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80109870 msgr2=0x7fad801111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.915+0000 7fad88000700 1 --2- 172.21.15.50:0/2349319433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80109870 0x7fad801111d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fad7800b0a0 tx=0x7fad7800b3b0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 -- 172.21.15.50:0/2349319433 shutdown_connections 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 --2- 172.21.15.50:0/2349319433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80109870 0x7fad801111d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 --2- 172.21.15.50:0/2349319433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad801045f0 0x7fad80104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 --2- 172.21.15.50:0/2349319433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80108f20 0x7fad80109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 -- 172.21.15.50:0/2349319433 >> 172.21.15.50:0/2349319433 conn(0x7fad80100040 msgr2=0x7fad80102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 -- 172.21.15.50:0/2349319433 shutdown_connections 2024-05-10T21:17:27.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.916+0000 7fad88000700 1 -- 172.21.15.50:0/2349319433 wait complete. 2024-05-10T21:17:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.917+0000 7fad88000700 1 Processor -- start 2024-05-10T21:17:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 -- start start 2024-05-10T21:17:27.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80108f20 0x7fad8019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad80109870 0x7fad8019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad801144a0 con 0x7fad80109870 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad80114320 con 0x7fad80108f20 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.918+0000 7fad88000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad80114620 con 0x7fad801045f0 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80108f20 0x7fad8019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41692/0 (socket says 172.21.15.50:41692) 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 -- 172.21.15.50:0/1764760241 learned_addr learned my addr 172.21.15.50:0/1764760241 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:27.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad86000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad80109870 0x7fad8019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 -- 172.21.15.50:0/1764760241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80108f20 msgr2=0x7fad8019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80108f20 0x7fad8019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 -- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad80109870 msgr2=0x7fad8019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad80109870 0x7fad8019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 -- 172.21.15.50:0/1764760241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8019fc80 con 0x7fad801045f0 2024-05-10T21:17:27.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.919+0000 7fad85600700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9a0 tx=0x7fad7400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.920+0000 7fad7e000700 1 -- 172.21.15.50:0/1764760241 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ca10 con 0x7fad801045f0 2024-05-10T21:17:27.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.920+0000 7fad7e000700 1 -- 172.21.15.50:0/1764760241 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad7400cbb0 con 0x7fad801045f0 2024-05-10T21:17:27.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.920+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8019ff70 con 0x7fad801045f0 2024-05-10T21:17:27.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.920+0000 7fad7e000700 1 -- 172.21.15.50:0/1764760241 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74018680 con 0x7fad801045f0 2024-05-10T21:17:27.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.920+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a7890 con 0x7fad801045f0 2024-05-10T21:17:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.924+0000 7fad7e000700 1 -- 172.21.15.50:0/1764760241 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fad74010430 con 0x7fad801045f0 2024-05-10T21:17:27.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.925+0000 7fad96ba0b80 1 Processor -- start 2024-05-10T21:17:27.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.925+0000 7fad7e000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 0x7fad680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 -- start start 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad7e000700 1 -- 172.21.15.50:0/1764760241 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fad740e2330 con 0x7fad801045f0 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad86000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 0x7fad680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 0x559d2fc82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc877a0 0x559d2fc87bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559d2fba0a20 con 0x559d2fc830a0 2024-05-10T21:17:27.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559d2fb3e760 con 0x559d2fc877a0 2024-05-10T21:17:27.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.926+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559d2fb9c9e0 con 0x559d2fb00e40 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 0x559d2fc82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad84c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad86000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 0x7fad680b4e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad78009900 tx=0x7fad7800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad84c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45494/0 (socket says 172.21.15.50:45494) 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad86000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc877a0 0x559d2fc87bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad84c00700 1 -- 172.21.15.50:0/2596317006 learned_addr learned my addr 172.21.15.50:0/2596317006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 -- 172.21.15.50:0/2596317006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc877a0 msgr2=0x559d2fc87bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc877a0 0x559d2fc87bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 -- 172.21.15.50:0/2596317006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 msgr2=0x559d2fc87260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad85600700 1 -- 172.21.15.50:0/2596317006 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2faed8d0 con 0x559d2fb00e40 2024-05-10T21:17:27.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.927+0000 7fad84c00700 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:27.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.928+0000 7fad85600700 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 0x559d2fc82b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e970 tx=0x7fad740b2140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a375d39c8b5af599 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.928+0000 7fad72a00700 1 -- 172.21.15.50:0/2596317006 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740aa020 con 0x559d2fb00e40 2024-05-10T21:17:27.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.928+0000 7fad72a00700 1 -- 172.21.15.50:0/2596317006 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad740a9920 con 0x559d2fb00e40 2024-05-10T21:17:27.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.928+0000 7fad72a00700 1 -- 172.21.15.50:0/2596317006 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740cabd0 con 0x559d2fb00e40 2024-05-10T21:17:27.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 -- 172.21.15.50:0/2596317006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 msgr2=0x559d2fc82b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 0x559d2fc82b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e970 tx=0x7fad740b2140 comp rx=0 tx=0).stop 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 -- 172.21.15.50:0/2596317006 shutdown_connections 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc877a0 0x559d2fc87bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fb00e40 0x559d2fc82b60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/2596317006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fc830a0 0x559d2fc87260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 -- 172.21.15.50:0/2596317006 >> 172.21.15.50:0/2596317006 conn(0x559d2fb45f70 msgr2=0x559d2fc792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.929+0000 7fad96ba0b80 1 -- 172.21.15.50:0/2596317006 shutdown_connections 2024-05-10T21:17:27.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.930+0000 7fad96ba0b80 1 -- 172.21.15.50:0/2596317006 wait complete. 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.930+0000 7fad96ba0b80 1 -- start start 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.930+0000 7fad96ba0b80 1 Processor -- start 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc8ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559d2fc8f690 con 0x559d2fb00e40 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559d2fb06b20 con 0x559d2fc830a0 2024-05-10T21:17:27.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559d2fc8f810 con 0x559d2fc877a0 2024-05-10T21:17:27.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.931+0000 7fad96ba0b80 10 client.0 _ll_register_callbacks cb 0x559d2fb47c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc8ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad85600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41700/0 (socket says 172.21.15.50:41700) 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 -- 172.21.15.50:0/607782529 learned_addr learned my addr 172.21.15.50:0/607782529 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc8ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41696/0 (socket says 172.21.15.50:41696) 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 -- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 msgr2=0x559d2fc8ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc8ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 -- 172.21.15.50:0/607782529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 msgr2=0x559d2fc8c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad86000700 1 -- 172.21.15.50:0/607782529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2fb66420 con 0x559d2fc877a0 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.932+0000 7fad85600700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:27.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.933+0000 7fad86000700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad7800b0a0 tx=0x7fad780099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51bf7e789f1392c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.933+0000 7fad67400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.933+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7802d070 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.933+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fad78007c20 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.934+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad740a9920 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.934+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fad78039050 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.934+0000 7fad96ba0b80 10 client.26746 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.934+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d2fc9a300 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.934+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad740a9920 con 0x559d2fc877a0 2024-05-10T21:17:27.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fad78011070 con 0x559d2fc877a0 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 20 client.26746 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fc9c8e0 0x559d2fc9ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 4 client.26746 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d2fc4b1c0 con 0x559d2fc9c8e0 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.935+0000 7fad85600700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fc9c8e0 0x559d2fc9ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.936+0000 7fad85600700 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fc9c8e0 0x559d2fc9ed20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88da2877b1c730f6 server_cookie=5297bf4286bdac0a in_seq=0 out_seq=0 2024-05-10T21:17:27.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.936+0000 7fad67400700 10 client.26746 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:27.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.938+0000 7fad67400700 1 -- 172.21.15.50:0/607782529 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d2fc4b1c0 con 0x559d2fc9c8e0 2024-05-10T21:17:27.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.938+0000 7fad67400700 10 client.26746 handle_command_reply: tid=0 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.938+0000 7fad96ba0b80 1 client.26746 shutdown 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.938+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 msgr2=0x559d2fc8cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.938+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad7800b0a0 tx=0x7fad780099e0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 shutdown_connections 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fc9c8e0 0x559d2fc9ed20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8cf40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/607782529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc8ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 >> 172.21.15.50:0/607782529 conn(0x559d2fb45f70 msgr2=0x559d2fc7d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad67400700 0 client.26746 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 shutdown_connections 2024-05-10T21:17:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 1 -- 172.21.15.50:0/607782529 wait complete. 2024-05-10T21:17:27.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.939+0000 7fad96ba0b80 20 client.26746 trim_cache size 0 max 16384 2024-05-10T21:17:27.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.956+0000 7fad96ba0b80 1 Processor -- start 2024-05-10T21:17:27.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.956+0000 7fad96ba0b80 1 -- start start 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad801af2c0 con 0x559d2fb00e40 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad801aca90 con 0x559d2fc830a0 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad801aa920 con 0x559d2fc877a0 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41712/0 (socket says 172.21.15.50:41712) 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad84c00700 1 -- 172.21.15.50:0/1919746890 learned_addr learned my addr 172.21.15.50:0/1919746890 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.957+0000 7fad85600700 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad86000700 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 -- 172.21.15.50:0/1919746890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 msgr2=0x559d2fc8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 -- 172.21.15.50:0/1919746890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 msgr2=0x559d2fc73ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc73ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 -- 172.21.15.50:0/1919746890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2fc8ed60 con 0x559d2fc830a0 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad84c00700 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0044e0 tx=0x7fad6c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec27d1eca3f2783c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.958+0000 7fad72000700 1 -- 172.21.15.50:0/1919746890 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004a30 con 0x559d2fc830a0 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.959+0000 7fad72000700 1 -- 172.21.15.50:0/1919746890 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6c0054d0 con 0x559d2fc830a0 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.959+0000 7fad72000700 1 -- 172.21.15.50:0/1919746890 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003c80 con 0x559d2fc830a0 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.959+0000 7fad96ba0b80 1 -- 172.21.15.50:0/1919746890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 msgr2=0x559d2fc88bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.959+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0044e0 tx=0x7fad6c009bf0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 -- 172.21.15.50:0/1919746890 shutdown_connections 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fc8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/1919746890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc88bc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 -- 172.21.15.50:0/1919746890 >> 172.21.15.50:0/1919746890 conn(0x559d2fb45f70 msgr2=0x559d2fc87060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 -- 172.21.15.50:0/1919746890 shutdown_connections 2024-05-10T21:17:27.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 -- 172.21.15.50:0/1919746890 wait complete. 2024-05-10T21:17:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.960+0000 7fad96ba0b80 1 -- start start 2024-05-10T21:17:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 Processor -- start 2024-05-10T21:17:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fd050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad801aaf70 con 0x559d2fb00e40 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ac3c0 con 0x559d2fc830a0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ab620 con 0x559d2fc877a0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad96ba0b80 10 client.0 _ll_register_callbacks cb 0x559d2fb47c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.961+0000 7fad85600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41722/0 (socket says 172.21.15.50:41722) 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 -- 172.21.15.50:0/3855398425 learned_addr learned my addr 172.21.15.50:0/3855398425 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad86000700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fd050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 -- 172.21.15.50:0/3855398425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 msgr2=0x559d2fd050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fd050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 -- 172.21.15.50:0/3855398425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 msgr2=0x559d2fc8fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad84c00700 1 -- 172.21.15.50:0/3855398425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2fd05600 con 0x559d2fc830a0 2024-05-10T21:17:27.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.962+0000 7fad85600700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad84c00700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad6c003780 tx=0x7fad6c003b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f45747c54b12c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad67400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004750 con 0x559d2fc830a0 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8177+0+0 (secure 0 0 0) 0x7fad6c0048f0 con 0x559d2fc830a0 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6c0054d0 con 0x559d2fc830a0 2024-05-10T21:17:27.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fad6c004b10 con 0x559d2fc830a0 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad96ba0b80 10 client.26807 fetch_fsmap learned FSMap version 37 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.963+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d2fd06780 con 0x559d2fc830a0 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6c0054d0 con 0x559d2fc830a0 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fad6c01a070 con 0x559d2fc830a0 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 10 client.26807 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 20 client.26807 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fd06ea0 0x559d2fd092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 4 client.26807 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.964+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d2fbae400 con 0x559d2fd06ea0 2024-05-10T21:17:27.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.965+0000 7fad85600700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fd06ea0 0x559d2fd092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.966+0000 7fad85600700 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fd06ea0 0x559d2fd092c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3ec11172f18932 server_cookie=5f82fafca24d6cb4 in_seq=0 out_seq=0 2024-05-10T21:17:27.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.966+0000 7fad67400700 10 client.26807 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:27.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.968+0000 7fad67400700 1 -- 172.21.15.50:0/3855398425 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d2fbae400 con 0x559d2fd06ea0 2024-05-10T21:17:27.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.968+0000 7fad67400700 10 client.26807 handle_command_reply: tid=0 2024-05-10T21:17:27.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.968+0000 7fad96ba0b80 1 client.26807 shutdown 2024-05-10T21:17:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 msgr2=0x559d2fc90260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad6c003780 tx=0x7fad6c003b10 comp rx=0 tx=0).stop 2024-05-10T21:17:27.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 shutdown_connections 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559d2fd06ea0 0x559d2fd092c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d2fc877a0 0x559d2fd050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d2fb00e40 0x559d2fc8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 --2- 172.21.15.50:0/3855398425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d2fc830a0 0x559d2fc90260 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 >> 172.21.15.50:0/3855398425 conn(0x559d2fb45f70 msgr2=0x559d2fc87060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad67400700 0 client.26807 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 shutdown_connections 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.969+0000 7fad96ba0b80 1 -- 172.21.15.50:0/3855398425 wait complete. 2024-05-10T21:17:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.970+0000 7fad96ba0b80 20 client.26807 trim_cache size 0 max 16384 2024-05-10T21:17:27.967 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:17:27.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.970+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 msgr2=0x7fad680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.970+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 0x7fad680b4e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad78009900 tx=0x7fad7800b720 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.970+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 msgr2=0x7fad8019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.970+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9a0 tx=0x7fad7400ecb0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 shutdown_connections 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fad680b2900 0x7fad680b4e10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad80109870 0x7fad8019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad801045f0 0x7fad8019adb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 --2- 172.21.15.50:0/1764760241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad80108f20 0x7fad8019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 >> 172.21.15.50:0/1764760241 conn(0x7fad80100040 msgr2=0x7fad80108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 shutdown_connections 2024-05-10T21:17:27.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:27.971+0000 7fad88000700 1 -- 172.21.15.50:0/1764760241 wait complete. 2024-05-10T21:17:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:28 smithi050 ceph-mon[29546]: daemon mds.h restarted 2024-05-10T21:17:28.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:28 smithi092 ceph-mon[34415]: daemon mds.h restarted 2024-05-10T21:17:28.365 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:17:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setuser ceph since I am not root 2024-05-10T21:17:28.365 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:17:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setgroup ceph since I am not root 2024-05-10T21:17:28.365 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:17:28 smithi092 ceph-mds[64505]: -- 172.21.15.92:0/875837011 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556c799da40 con 0x5556c79e5600 2024-05-10T21:17:28.366 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:17:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: starting mds.h at 2024-05-10T21:17:28.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:28 smithi142 ceph-mon[34846]: daemon mds.h restarted 2024-05-10T21:17:29.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: pgmap v719: 97 pgs: 97 active+clean; 476 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 661 KiB/s wr, 230 op/s 2024-05-10T21:17:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T21:17:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6836/1239523786,v1:172.21.15.92:6837/1239523786] up:boot 2024-05-10T21:17:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:17:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:17:29.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:29 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: pgmap v719: 97 pgs: 97 active+clean; 476 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 661 KiB/s wr, 230 op/s 2024-05-10T21:17:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T21:17:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6836/1239523786,v1:172.21.15.92:6837/1239523786] up:boot 2024-05-10T21:17:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:17:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:17:29.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:29 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: pgmap v719: 97 pgs: 97 active+clean; 476 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 661 KiB/s wr, 230 op/s 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6836/1239523786,v1:172.21.15.92:6837/1239523786] up:boot 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:17:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:29 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:30.193 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T21:17:30.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T21:17:30.688 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:31 smithi050 ceph-mon[29546]: pgmap v721: 97 pgs: 97 active+clean; 471 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 495 KiB/s wr, 206 op/s 2024-05-10T21:17:31.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:31 smithi092 ceph-mon[34415]: pgmap v721: 97 pgs: 97 active+clean; 471 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 495 KiB/s wr, 206 op/s 2024-05-10T21:17:31.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:31 smithi142 ceph-mon[34846]: pgmap v721: 97 pgs: 97 active+clean; 471 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 495 KiB/s wr, 206 op/s 2024-05-10T21:17:33.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:33 smithi092 ceph-mon[34415]: pgmap v722: 97 pgs: 97 active+clean; 455 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 869 KiB/s wr, 220 op/s 2024-05-10T21:17:33.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.368+0000 7f7f29a00700 1 -- 172.21.15.50:0/828752599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24079750 msgr2=0x7f7f24114cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.368+0000 7f7f29a00700 1 --2- 172.21.15.50:0/828752599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24079750 0x7f7f24114cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7f100098f0 tx=0x7f7f10009c00 comp rx=0 tx=0).stop 2024-05-10T21:17:33.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 -- 172.21.15.50:0/828752599 shutdown_connections 2024-05-10T21:17:33.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 --2- 172.21.15.50:0/828752599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 0x7f7f24117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 --2- 172.21.15.50:0/828752599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24078e00 0x7f7f24079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 --2- 172.21.15.50:0/828752599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24079750 0x7f7f24114cc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 -- 172.21.15.50:0/828752599 >> 172.21.15.50:0/828752599 conn(0x7f7f24102080 msgr2=0x7f7f241044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.366 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-mon[29546]: pgmap v722: 97 pgs: 97 active+clean; 455 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 869 KiB/s wr, 220 op/s 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 -- 172.21.15.50:0/828752599 shutdown_connections 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.369+0000 7f7f29a00700 1 -- 172.21.15.50:0/828752599 wait complete. 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.370+0000 7f7f29a00700 1 Processor -- start 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 -- start start 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24079750 0x7f7f241a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 0x7f7f241a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2411aa70 con 0x7f7f24079750 2024-05-10T21:17:33.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2411a8f0 con 0x7f7f24078e00 2024-05-10T21:17:33.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f29a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2411abf0 con 0x7f7f24115200 2024-05-10T21:17:33.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f22a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f22a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41732/0 (socket says 172.21.15.50:41732) 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f22a00700 1 -- 172.21.15.50:0/2214727701 learned_addr learned my addr 172.21.15.50:0/2214727701 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.371+0000 7f7f22000700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24079750 0x7f7f241a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f23400700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 0x7f7f241a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 -- 172.21.15.50:0/2214727701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 msgr2=0x7f7f241a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 0x7f7f241a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 -- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24079750 msgr2=0x7f7f241a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24079750 0x7f7f241a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 -- 172.21.15.50:0/2214727701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f241a6370 con 0x7f7f24078e00 2024-05-10T21:17:33.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f22a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400ec10 tx=0x7f7f1400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.372+0000 7f7f1b400700 1 -- 172.21.15.50:0/2214727701 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400cc90 con 0x7f7f24078e00 2024-05-10T21:17:33.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.373+0000 7f7f1b400700 1 -- 172.21.15.50:0/2214727701 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1400ce30 con 0x7f7f24078e00 2024-05-10T21:17:33.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.373+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241a6660 con 0x7f7f24078e00 2024-05-10T21:17:33.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.373+0000 7f7f1b400700 1 -- 172.21.15.50:0/2214727701 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14010430 con 0x7f7f24078e00 2024-05-10T21:17:33.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.373+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241adf80 con 0x7f7f24078e00 2024-05-10T21:17:33.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.378+0000 7f7f1b400700 1 -- 172.21.15.50:0/2214727701 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f14004000 con 0x7f7f24078e00 2024-05-10T21:17:33.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.378+0000 7f7f384c7b80 1 Processor -- start 2024-05-10T21:17:33.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 -- start start 2024-05-10T21:17:33.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:33 smithi142 ceph-mon[34846]: pgmap v722: 97 pgs: 97 active+clean; 455 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 869 KiB/s wr, 220 op/s 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7281370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f727b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f727be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5581f719daa0 con 0x5581f727ba70 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5581f713b7e0 con 0x5581f72818b0 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581f7199a60 con 0x5581f70fdec0 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f727b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.379+0000 7f7f1b400700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 0x7f7f080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f23400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7281370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f23400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7281370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41740/0 (socket says 172.21.15.50:41740) 2024-05-10T21:17:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f23400700 1 -- 172.21.15.50:0/3745419830 learned_addr learned my addr 172.21.15.50:0/3745419830 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22000700 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f727be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f1b400700 1 -- 172.21.15.50:0/2214727701 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f7f140e2430 con 0x7f7f24078e00 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 -- 172.21.15.50:0/3745419830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 msgr2=0x5581f7281370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7281370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 -- 172.21.15.50:0/3745419830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 msgr2=0x5581f727be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f727be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 -- 172.21.15.50:0/3745419830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f70ea950 con 0x5581f72818b0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f23400700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 0x7f7f080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22000700 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f727be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.380+0000 7f7f22a00700 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f727b530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f140dfea0 tx=0x7f7f1400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52339a102ac5276 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.381+0000 7f7f0fe00700 1 -- 172.21.15.50:0/3745419830 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140a9190 con 0x5581f72818b0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.381+0000 7f7f0fe00700 1 -- 172.21.15.50:0/3745419830 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f140a9530 con 0x5581f72818b0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.381+0000 7f7f0fe00700 1 -- 172.21.15.50:0/3745419830 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140b2880 con 0x5581f72818b0 2024-05-10T21:17:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3745419830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 msgr2=0x5581f727b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f23400700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 0x7f7f080b4e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009790 tx=0x7f7f1c004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f727b530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f140dfea0 tx=0x7f7f1400ef90 comp rx=0 tx=0).stop 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3745419830 shutdown_connections 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f727be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7281370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3745419830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f727b530 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3745419830 >> 172.21.15.50:0/3745419830 conn(0x5581f7142ff0 msgr2=0x5581f7276350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3745419830 shutdown_connections 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.382+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3745419830 wait complete. 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.383+0000 7f7f384c7b80 1 -- start start 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.383+0000 7f7f384c7b80 1 Processor -- start 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f72855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5581f7288330 con 0x5581f727ba70 2024-05-10T21:17:33.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5581f7103ba0 con 0x5581f72818b0 2024-05-10T21:17:33.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581f72884b0 con 0x5581f70fdec0 2024-05-10T21:17:33.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f384c7b80 10 client.0 _ll_register_callbacks cb 0x5581f7144cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f22a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41744/0 (socket says 172.21.15.50:41744) 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f22a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41746/0 (socket says 172.21.15.50:41746) 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 -- 172.21.15.50:0/816892673 learned_addr learned my addr 172.21.15.50:0/816892673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f22000700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f72855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 -- 172.21.15.50:0/816892673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 msgr2=0x5581f7285070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 -- 172.21.15.50:0/816892673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 msgr2=0x5581f72855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f72855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f23400700 1 -- 172.21.15.50:0/816892673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f71634a0 con 0x5581f72818b0 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f22000700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f72855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:33.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.384+0000 7f7f22a00700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f23400700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007710 tx=0x7f7f1c009130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aaa24fe261dfe9b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c028060 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f7f1c017660 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f140a9530 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c017860 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f384c7b80 10 client.26825 fetch_fsmap learned FSMap version 39 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.385+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5581f7292f30 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f140a9530 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7f1c02eba0 con 0x5581f72818b0 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 10 client.26825 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 20 client.26825 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7295570 0x5581f72979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 4 client.26825 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:33.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581f7248240 con 0x5581f7295570 2024-05-10T21:17:33.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.386+0000 7f7f22a00700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7295570 0x5581f72979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.387+0000 7f7f22a00700 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7295570 0x5581f72979b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ee37dff1747302 server_cookie=8b1e609f423ba534 in_seq=0 out_seq=0 2024-05-10T21:17:33.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.387+0000 7f7f0cc00700 10 client.26825 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.388+0000 7f7f0cc00700 1 -- 172.21.15.50:0/816892673 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581f7248240 con 0x5581f7295570 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.388+0000 7f7f0cc00700 10 client.26825 handle_command_reply: tid=0 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.388+0000 7f7f384c7b80 1 client.26825 shutdown 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 msgr2=0x5581f7285af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007710 tx=0x7f7f1c009130 comp rx=0 tx=0).stop 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 shutdown_connections 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7295570 0x5581f72979b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7285af0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f7285070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/816892673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f727ba70 0x5581f72855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 >> 172.21.15.50:0/816892673 conn(0x5581f7142ff0 msgr2=0x5581f727a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f0cc00700 0 client.26825 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:33.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 shutdown_connections 2024-05-10T21:17:33.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 1 -- 172.21.15.50:0/816892673 wait complete. 2024-05-10T21:17:33.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.389+0000 7f7f384c7b80 20 client.26825 trim_cache size 0 max 16384 2024-05-10T21:17:33.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.417+0000 7f7f384c7b80 1 Processor -- start 2024-05-10T21:17:33.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.417+0000 7f7f384c7b80 1 -- start start 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f70fdec0 0x5581f727b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7284240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b7c40 con 0x5581f70fdec0 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b5410 con 0x5581f72818b0 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b32c0 con 0x5581f727ba70 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f22000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f23400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7284240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f22000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41762/0 (socket says 172.21.15.50:41762) 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f22000700 1 -- 172.21.15.50:0/3345903516 learned_addr learned my addr 172.21.15.50:0/3345903516 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.418+0000 7f7f22a00700 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f70fdec0 0x5581f727b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 -- 172.21.15.50:0/3345903516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 msgr2=0x5581f7284240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7284240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 -- 172.21.15.50:0/3345903516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f70fdec0 msgr2=0x5581f727b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f70fdec0 0x5581f727b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 -- 172.21.15.50:0/3345903516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f726e460 con 0x5581f727ba70 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f22000700 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7f100098f0 tx=0x7f7f10005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f62b51a5535ad7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f0f400700 1 -- 172.21.15.50:0/3345903516 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000f660 con 0x5581f727ba70 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f0f400700 1 -- 172.21.15.50:0/3345903516 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1000f800 con 0x5581f727ba70 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.419+0000 7f7f0f400700 1 -- 172.21.15.50:0/3345903516 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10003ae0 con 0x5581f727ba70 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.420+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3345903516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 msgr2=0x5581f727f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.420+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7f100098f0 tx=0x7f7f10005dc0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3345903516 shutdown_connections 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f72818b0 0x5581f7284240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f70fdec0 0x5581f727b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/3345903516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f727ba70 0x5581f727f760 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3345903516 >> 172.21.15.50:0/3345903516 conn(0x5581f7142ff0 msgr2=0x5581f7270120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3345903516 shutdown_connections 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 -- 172.21.15.50:0/3345903516 wait complete. 2024-05-10T21:17:33.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 -- start start 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.421+0000 7f7f384c7b80 1 Processor -- start 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f728d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f72818b0 0x5581f728ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b3910 con 0x5581f72818b0 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b4d40 con 0x5581f727ba70 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241b3fa0 con 0x5581f70fdec0 2024-05-10T21:17:33.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f384c7b80 10 client.0 _ll_register_callbacks cb 0x5581f7144cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.422+0000 7f7f22000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:41752/0 (socket says 172.21.15.50:41752) 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 -- 172.21.15.50:0/324247182 learned_addr learned my addr 172.21.15.50:0/324247182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f23400700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f72818b0 0x5581f728ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22a00700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f728d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 -- 172.21.15.50:0/324247182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 msgr2=0x5581f728d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f728d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 -- 172.21.15.50:0/324247182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f72818b0 msgr2=0x5581f728ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f72818b0 0x5581f728ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 -- 172.21.15.50:0/324247182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f728b1b0 con 0x5581f727ba70 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22a00700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f728d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f22000700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f1000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1dce50cd3124e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.423+0000 7f7f0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000f9b0 con 0x5581f727ba70 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1000f800 con 0x5581f727ba70 2024-05-10T21:17:33.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f7f1000bae0 con 0x5581f727ba70 2024-05-10T21:17:33.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f1000bce0 con 0x5581f727ba70 2024-05-10T21:17:33.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f384c7b80 10 client.26831 fetch_fsmap learned FSMap version 39 2024-05-10T21:17:33.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5581f7300930 con 0x5581f727ba70 2024-05-10T21:17:33.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.424+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1000f800 con 0x5581f727ba70 2024-05-10T21:17:33.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7f1002b440 con 0x5581f727ba70 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 10 client.26831 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 10 client.26831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 20 client.26831 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7300fc0 0x5581f73033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 4 client.26831 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581f71ab480 con 0x5581f7300fc0 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.425+0000 7f7f22a00700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7300fc0 0x5581f73033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.426+0000 7f7f22a00700 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7300fc0 0x5581f73033e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b58167dfb1af02 server_cookie=82f5017fba98ff3a in_seq=0 out_seq=0 2024-05-10T21:17:33.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.426+0000 7f7f0cc00700 10 client.26831 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f0cc00700 1 -- 172.21.15.50:0/324247182 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581f71ab480 con 0x5581f7300fc0 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f0cc00700 10 client.26831 handle_command_reply: tid=0 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 client.26831 shutdown 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 msgr2=0x5581f728a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f1000b810 comp rx=0 tx=0).stop 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 shutdown_connections 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x5581f7300fc0 0x5581f73033e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581f72818b0 0x5581f728ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581f70fdec0 0x5581f728d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 --2- 172.21.15.50:0/324247182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5581f727ba70 0x5581f728a730 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f0cc00700 0 client.26831 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.427+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 >> 172.21.15.50:0/324247182 conn(0x5581f7142ff0 msgr2=0x5581f727f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.428+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 shutdown_connections 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.428+0000 7f7f384c7b80 1 -- 172.21.15.50:0/324247182 wait complete. 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.428+0000 7f7f384c7b80 20 client.26831 trim_cache size 0 max 16384 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stdout: "return_code": 0, 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_tag": "1fe31dca-6c29-4c80-8d3f-5a2cf3d85748", 2024-05-10T21:17:33.425 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": "asynchronous" 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 msgr2=0x7f7f080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 0x7f7f080b4e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009790 tx=0x7f7f1c004ad0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 msgr2=0x7f7f241a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400ec10 tx=0x7f7f1400ef20 comp rx=0 tx=0).stop 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 shutdown_connections 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f7f080b2900 0x7f7f080b4e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24078e00 0x7f7f241a14a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f24115200 0x7f7f241a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 --2- 172.21.15.50:0/2214727701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f24079750 0x7f7f241a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:33.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 >> 172.21.15.50:0/2214727701 conn(0x7f7f24102080 msgr2=0x7f7f2407c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:33.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 shutdown_connections 2024-05-10T21:17:33.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:33.429+0000 7f7f29a00700 1 -- 172.21.15.50:0/2214727701 wait complete. 2024-05-10T21:17:33.664 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.531+0000 7f885a200700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000bd9 [...2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29/ auth{2=1} fragtree_t(*^3) v88943 ap=2 DIRTYPARENT f(v4 m2024-05-10T21:13:29.828217+0000 134=134+0) n(v6 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1) (inest sync->mix g=2) caps={14925=pAsLsXsFs/pAsLsXsFsx@162} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55ff492d8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000000bd9:[<0x10000000399/blog-29 v88943>,<0x10000000208/blogtest_in v2287>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T21:12:38.539504+0000 133=133+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1)","memory_value.dirstat":"f(v4 m2024-05-10T21:13:29.828217+0000 134=134+0)","memory_value.rstat":"n(v6 rc2024-05-10T21:13:29.828217+0000 b1748221 135=134+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.674+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000287a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml auth v995 ap=1 s=22510 n(v0 rc2024-05-10T21:15:38.690011+0000 b22510 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48f99700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000287a:[<0x20000002873/article-59.xml v902>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000287a:[<0x20000002873/article-59.xml v995>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.675+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a2b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg auth v971 ap=1 s=16634 n(v0 rc2024-05-10T21:15:41.646938+0000 b16634 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48f98c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a2b:[<0x20000002873/picture-13.jpg v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a2b:[<0x20000002873/picture-13.jpg v971>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.675+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a1c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg auth v975 ap=1 s=47517 n(v0 rc2024-05-10T21:15:41.617939+0000 b47517 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ceb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a1c:[<0x20000002873/picture-18.jpg v937>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a1c:[<0x20000002873/picture-18.jpg v975>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.676+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002901 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml auth v977 ap=1 s=23391 n(v0 rc2024-05-10T21:15:39.180999+0000 b23391 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cec100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002901:[<0x20000002873/article-18.xml v904>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002901:[<0x20000002873/article-18.xml v977>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.677+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a19 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg auth v979 ap=1 s=18239 n(v0 rc2024-05-10T21:15:41.611939+0000 b18239 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cecc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a19:[<0x20000002873/picture-19.jpg v939>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a19:[<0x20000002873/picture-19.jpg v979>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.678+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002897 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml auth v993 ap=1 s=17033 n(v0 rc2024-05-10T21:15:38.754010+0000 b17033 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ced180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002897:[<0x20000002873/article-51.xml v906>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002897:[<0x20000002873/article-51.xml v993>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028e2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml auth v983 ap=1 s=27830 n(v0 rc2024-05-10T21:15:39.039003+0000 b27830 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff492f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028e2:[<0x20000002873/article-27.xml v908>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028e2:[<0x20000002873/article-27.xml v983>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a46 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg auth v965 ap=1 s=24537 n(v0 rc2024-05-10T21:15:41.699937+0000 b24537 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff492f4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a46:[<0x20000002873/picture-4.jpg v953>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a46:[<0x20000002873/picture-4.jpg v965>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.941 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.680+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029e9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg auth v987 ap=1 s=24613 n(v0 rc2024-05-10T21:15:41.488942+0000 b24613 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff492f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029e9:[<0x20000002873/picture-35.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029e9:[<0x20000002873/picture-35.jpg v987>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.942 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.681+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a4c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg auth v963 ap=1 s=56967 n(v0 rc2024-05-10T21:15:41.711936+0000 b56967 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff492f6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a4c:[<0x20000002873/picture-2.jpg v951>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a4c:[<0x20000002873/picture-2.jpg v963>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.942 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.681+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028fa [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml auth v981 ap=1 s=21456 n(v0 rc2024-05-10T21:15:39.150000+0000 b21456 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a6e3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028fa:[<0x20000002873/article-20.xml v910>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028fa:[<0x20000002873/article-20.xml v981>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.942 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.681+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a28 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg auth v973 ap=1 s=47236 n(v0 rc2024-05-10T21:15:41.640938+0000 b47236 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a6e3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a28:[<0x20000002873/picture-14.jpg v955>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a28:[<0x20000002873/picture-14.jpg v973>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.942 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028b7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml auth v989 ap=1 s=3551 n(v0 rc2024-05-10T21:15:38.871007+0000 b3551 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a6e5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028b7:[<0x20000002873/article-41.xml v912>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028b7:[<0x20000002873/article-41.xml v989>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.942 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028aa [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml auth v991 ap=1 s=27456 n(v0 rc2024-05-10T21:15:38.827008+0000 b27456 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ae0a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028aa:[<0x20000002873/article-45.xml v914>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028aa:[<0x20000002873/article-45.xml v991>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a3d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg auth v967 ap=1 s=41203 n(v0 rc2024-05-10T21:15:41.681937+0000 b41203 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ae09b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a3d:[<0x20000002873/picture-7.jpg v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a3d:[<0x20000002873/picture-7.jpg v967>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002973 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg auth v997 ap=1 s=27184 n(v0 rc2024-05-10T21:15:41.104951+0000 b27184 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ae0b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002973:[<0x20000002873/picture-61.jpg v941>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002973:[<0x20000002873/picture-61.jpg v997>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.683+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028d4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml auth v985 ap=1 s=31725 n(v0 rc2024-05-10T21:15:38.988004+0000 b31725 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ae09080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028d4:[<0x20000002873/article-31.xml v916>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028d4:[<0x20000002873/article-31.xml v985>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.684+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000296a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg auth v969 ap=1 s=23101 n(v0 rc2024-05-10T21:15:41.060953+0000 b23101 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b377700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000296a:[<0x20000002873/picture-64.jpg v969>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000296a:[<0x20000002873/picture-64.jpg v969>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.684+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a37 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg auth v969 ap=1 s=14732 n(v0 rc2024-05-10T21:15:41.669937+0000 b14732 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ae08580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a37:[<0x20000002873/picture-9.jpg v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a37:[<0x20000002873/picture-9.jpg v969>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a34 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg auth v991 ap=1 s=53307 n(v0 rc2024-05-10T21:15:41.664938+0000 b53307 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49391600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a34:[<0x20000002873/picture-10.jpg v973>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a34:[<0x20000002873/picture-10.jpg v991>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a4f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg auth v981 ap=1 s=13702 n(v0 rc2024-05-10T21:15:41.717936+0000 b13702 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49392c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a4f:[<0x20000002873/picture-1.jpg v957>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a4f:[<0x20000002873/picture-1.jpg v981>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.943 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028ae [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml auth v1009 ap=1 s=3795 n(v0 rc2024-05-10T21:15:38.837008+0000 b3795 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49391b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028ae:[<0x20000002873/article-44.xml v898>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028ae:[<0x20000002873/article-44.xml v1009>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002924 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml auth v989 ap=1 s=13310 n(v0 rc2024-05-10T21:15:39.381994+0000 b13310 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49393180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002924:[<0x20000002873/article-9.xml v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002924:[<0x20000002873/article-9.xml v989>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.686+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a40 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg auth v985 ap=1 s=48563 n(v0 rc2024-05-10T21:15:41.687937+0000 b48563 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a44000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a40:[<0x20000002873/picture-6.jpg v967>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a40:[<0x20000002873/picture-6.jpg v985>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.686+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028ec [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml auth v997 ap=1 s=6507 n(v0 rc2024-05-10T21:15:39.083001+0000 b6507 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a46100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028ec:[<0x20000002873/article-24.xml v900>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028ec:[<0x20000002873/article-24.xml v997>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.686+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029fe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg auth v999 ap=1 s=22159 n(v0 rc2024-05-10T21:15:41.555940+0000 b22159 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a45080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029fe:[<0x20000002873/picture-28.jpg v953>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029fe:[<0x20000002873/picture-28.jpg v999>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.686+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029e3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg auth v1003 ap=1 s=21526 n(v0 rc2024-05-10T21:15:41.468942+0000 b21526 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4866a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029e3:[<0x20000002873/picture-37.jpg v961>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029e3:[<0x20000002873/picture-37.jpg v1003>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.687+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000293a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml auth v983 ap=1 s=1242 n(v0 rc2024-05-10T21:15:40.842958+0000 b1242 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49237080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000293a:[<0x20000002873/article-3.xml v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000293a:[<0x20000002873/article-3.xml v983>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.687+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a0a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg auth v995 ap=1 s=48055 n(v0 rc2024-05-10T21:15:41.582940+0000 b48055 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49236b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a0a:[<0x20000002873/picture-24.jpg v951>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a0a:[<0x20000002873/picture-24.jpg v995>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.687+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a1f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg auth v993 ap=1 s=50830 n(v0 rc2024-05-10T21:15:41.623939+0000 b50830 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4866a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a1f:[<0x20000002873/picture-17.jpg v971>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a1f:[<0x20000002873/picture-17.jpg v993>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.687+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000297f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg auth v1011 ap=1 s=61061 n(v0 rc2024-05-10T21:15:41.159950+0000 b61061 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff481a3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000297f:[<0x20000002873/picture-57.jpg v955>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000297f:[<0x20000002873/picture-57.jpg v1011>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.944 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000292c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml auth v987 ap=1 s=3469 n(v0 rc2024-05-10T21:15:40.740960+0000 b3469 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff481a4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000292c:[<0x20000002873/article-7.xml v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000292c:[<0x20000002873/article-7.xml v987>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.945 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000295e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg auth v947 ap=1 s=26430 n(v0 rc2024-05-10T21:15:41.009954+0000 b26430 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff478a3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000295e:[<0x20000002873/picture-68.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000295e:[<0x20000002873/picture-68.jpg v947>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.945 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029ef [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg auth v1001 ap=1 s=26113 n(v0 rc2024-05-10T21:15:41.507941+0000 b26113 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49238100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029ef:[<0x20000002873/picture-33.jpg v959>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029ef:[<0x20000002873/picture-33.jpg v1001>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.945 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029cd [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg auth v1007 ap=1 s=57879 n(v0 rc2024-05-10T21:15:41.416944+0000 b57879 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff481a5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029cd:[<0x20000002873/picture-42.jpg v965>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029cd:[<0x20000002873/picture-42.jpg v1007>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.945 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029dc [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg auth v1005 ap=1 s=23018 n(v0 rc2024-05-10T21:15:41.447943+0000 b23018 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff478a1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029dc:[<0x20000002873/picture-39.jpg v963>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029dc:[<0x20000002873/picture-39.jpg v1005>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.688+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a07 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg auth v981 ap=1 s=44794 n(v0 rc2024-05-10T21:15:41.576940+0000 b44794 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49c41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a07:[<0x20000002873/picture-25.jpg v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a07:[<0x20000002873/picture-25.jpg v981>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002889 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml auth v997 ap=1 s=16990 n(v0 rc2024-05-10T21:15:38.721010+0000 b16990 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47ff2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002889:[<0x20000002873/article-55.xml v876>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002889:[<0x20000002873/article-55.xml v997>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002928 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml auth v975 ap=1 s=11559 n(v0 rc2024-05-10T21:15:39.559990+0000 b11559 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4783b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002928:[<0x20000002873/article-8.xml v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002928:[<0x20000002873/article-8.xml v975>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029aa [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg auth v991 ap=1 s=42130 n(v0 rc2024-05-10T21:15:41.345945+0000 b42130 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4783c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029aa:[<0x20000002873/picture-49.jpg v959>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029aa:[<0x20000002873/picture-49.jpg v991>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029f5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg auth v989 ap=1 s=25186 n(v0 rc2024-05-10T21:15:41.527941+0000 b25186 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4783d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029f5:[<0x20000002873/picture-31.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029f5:[<0x20000002873/picture-31.jpg v989>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.690+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dab [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml auth v1001 ap=1 s=16531 n(v0 rc2024-05-10T21:15:52.207677+0000 b16531 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48be5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dab:[<0x20000002873/article-32.xml v927>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000000dab:[<0x20000002873/article-32.xml v1001>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.690+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a3a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg auth v973 ap=1 s=64593 n(v0 rc2024-05-10T21:15:41.676937+0000 b64593 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48be5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a3a:[<0x20000002873/picture-8.jpg v955>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a3a:[<0x20000002873/picture-8.jpg v973>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.690+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028f7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml auth v979 ap=1 s=10683 n(v0 rc2024-05-10T21:15:39.134000+0000 b10683 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49f7ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028f7:[<0x20000002873/article-21.xml v880>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028f7:[<0x20000002873/article-21.xml v979>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028d8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml auth v987 ap=1 s=32327 n(v0 rc2024-05-10T21:15:39.000004+0000 b32327 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49f7cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028d8:[<0x20000002873/article-30.xml v882>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028d8:[<0x20000002873/article-30.xml v987>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002914 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml auth v977 ap=1 s=20828 n(v0 rc2024-05-10T21:15:39.274997+0000 b20828 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49f7c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002914:[<0x20000002873/article-13.xml v884>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002914:[<0x20000002873/article-13.xml v977>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a04 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg auth v983 ap=1 s=20088 n(v0 rc2024-05-10T21:15:41.570940+0000 b20088 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48be4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a04:[<0x20000002873/picture-26.jpg v951>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a04:[<0x20000002873/picture-26.jpg v983>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002930 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml auth v971 ap=1 s=15223 n(v0 rc2024-05-10T21:15:40.775960+0000 b15223 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a1a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002930:[<0x20000002873/article-6.xml v941>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002930:[<0x20000002873/article-6.xml v971>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000293d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml auth v969 ap=1 s=1388 n(v0 rc2024-05-10T21:15:40.863957+0000 b1388 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a19600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000293d:[<0x20000002873/article-2.xml v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000293d:[<0x20000002873/article-2.xml v969>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000289b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml auth v993 ap=1 s=6325 n(v0 rc2024-05-10T21:15:38.763009+0000 b6325 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49f7c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000289b:[<0x20000002873/article-50.xml v886>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000289b:[<0x20000002873/article-50.xml v993>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029fb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg auth v985 ap=1 s=35929 n(v0 rc2024-05-10T21:15:41.546940+0000 b35929 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a1a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029fb:[<0x20000002873/picture-29.jpg v953>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029fb:[<0x20000002873/picture-29.jpg v985>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000298d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg auth v995 ap=1 s=11137 n(v0 rc2024-05-10T21:15:41.244948+0000 b11137 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a19080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000298d:[<0x20000002873/picture-54.jpg v957>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000298d:[<0x20000002873/picture-54.jpg v995>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002885 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml auth v999 ap=1 s=19109 n(v0 rc2024-05-10T21:15:38.713011+0000 b19109 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49d39080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002885:[<0x20000002873/article-56.xml v838>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002885:[<0x20000002873/article-56.xml v999>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028b1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml auth v970 ap=1 s=3960 n(v0 rc2024-05-10T21:15:38.850007+0000 b3960 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49d3ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028b1:[<0x20000002873/article-43.xml v840>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028b1:[<0x20000002873/article-43.xml v970>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.693+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a43 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg auth v964 ap=1 s=18946 n(v0 rc2024-05-10T21:15:41.693937+0000 b18946 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49d38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a43:[<0x20000002873/picture-5.jpg v952>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a43:[<0x20000002873/picture-5.jpg v964>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.693+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029a5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg auth v974 ap=1 s=35033 n(v0 rc2024-05-10T21:15:41.321946+0000 b35033 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a43180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029a5:[<0x20000002873/picture-50.jpg v946>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029a5:[<0x20000002873/picture-50.jpg v974>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.693+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028ce [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml auth v968 ap=1 s=27743 n(v0 rc2024-05-10T21:15:38.953005+0000 b27743 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49d3b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028ce:[<0x20000002873/article-33.xml v842>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028ce:[<0x20000002873/article-33.xml v968>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.693+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000297c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg auth v984 ap=1 s=18583 n(v0 rc2024-05-10T21:15:41.143950+0000 b18583 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000297c:[<0x20000002873/picture-58.jpg v948>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000297c:[<0x20000002873/picture-58.jpg v984>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.693+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002881 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml auth v982 ap=1 s=21003 n(v0 rc2024-05-10T21:15:38.707011+0000 b21003 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a40580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002881:[<0x20000002873/article-57.xml v844>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002881:[<0x20000002873/article-57.xml v982>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002967 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg auth v938 ap=1 s=54606 n(v0 rc2024-05-10T21:15:41.046953+0000 b54606 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49a42680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002967:[<0x20000002873/picture-65.jpg v938>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002967:[<0x20000002873/picture-65.jpg v938>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002890 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml auth v980 ap=1 s=19357 n(v0 rc2024-05-10T21:15:38.736010+0000 b19357 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49774c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002890:[<0x20000002873/article-53.xml v824>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002890:[<0x20000002873/article-53.xml v980>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000287e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml auth v986 ap=1 s=3785 n(v0 rc2024-05-10T21:15:38.698011+0000 b3785 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49773080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000287e:[<0x20000002873/article-58.xml v846>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000287e:[<0x20000002873/article-58.xml v986>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028df [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml auth v966 ap=1 s=20322 n(v0 rc2024-05-10T21:15:39.026003+0000 b20322 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49775700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028df:[<0x20000002873/article-28.xml v826>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028df:[<0x20000002873/article-28.xml v966>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002894 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml auth v978 ap=1 s=3811 n(v0 rc2024-05-10T21:15:38.745010+0000 b3811 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49772000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002894:[<0x20000002873/article-52.xml v828>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002894:[<0x20000002873/article-52.xml v978>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.695+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002943 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg auth v944 ap=1 s=11769 n(v0 rc2024-05-10T21:15:40.903956+0000 b11769 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4a178680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002943:[<0x20000002873/picture-77.jpg v944>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002943:[<0x20000002873/picture-77.jpg v944>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.695+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000295b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg auth v950 ap=1 s=10360 n(v0 rc2024-05-10T21:15:40.998954+0000 b10360 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a179180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000295b:[<0x20000002873/picture-69.jpg v950>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000295b:[<0x20000002873/picture-69.jpg v950>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.695+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029b9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg auth v972 ap=1 s=58141 n(v0 rc2024-05-10T21:15:41.376945+0000 b58141 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48258680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029b9:[<0x20000002873/picture-46.jpg v942>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029b9:[<0x20000002873/picture-46.jpg v972>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.695+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000299f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg auth v976 ap=1 s=24771 n(v0 rc2024-05-10T21:15:41.300947+0000 b24771 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a176b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000299f:[<0x20000002873/picture-51.jpg v940>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000299f:[<0x20000002873/picture-51.jpg v976>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.696+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002970 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg auth v936 ap=1 s=34869 n(v0 rc2024-05-10T21:15:41.088952+0000 b34869 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a177b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002970:[<0x20000002873/picture-62.jpg v936>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002970:[<0x20000002873/picture-62.jpg v936>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.696+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a49 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg auth v957 ap=1 s=64390 n(v0 rc2024-05-10T21:15:41.705937+0000 b64390 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a11cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a49:[<0x20000002873/picture-3.jpg v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a49:[<0x20000002873/picture-3.jpg v957>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.951 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.696+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002985 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg auth v973 ap=1 s=17993 n(v0 rc2024-05-10T21:15:41.217949+0000 b17993 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a11e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002985:[<0x20000002873/picture-55.jpg v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002985:[<0x20000002873/picture-55.jpg v973>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.696+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a31 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg auth v961 ap=1 s=56828 n(v0 rc2024-05-10T21:15:41.658938+0000 b56828 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a11d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a31:[<0x20000002873/picture-11.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a31:[<0x20000002873/picture-11.jpg v961>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.697+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002958 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg auth v935 ap=1 s=50836 n(v0 rc2024-05-10T21:15:40.987954+0000 b50836 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a11c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002958:[<0x20000002873/picture-70.jpg v935>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002958:[<0x20000002873/picture-70.jpg v935>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.697+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002934 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml auth v959 ap=1 s=19355 n(v0 rc2024-05-10T21:15:40.803959+0000 b19355 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a11db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002934:[<0x20000002873/article-5.xml v941>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002934:[<0x20000002873/article-5.xml v959>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.698+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028cc [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml auth v967 ap=1 s=5761 n(v0 rc2024-05-10T21:15:38.944005+0000 b5761 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49894000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028cc:[<0x20000002873/article-34.xml v888>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028cc:[<0x20000002873/article-34.xml v967>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.698+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002955 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg auth v939 ap=1 s=8699 n(v0 rc2024-05-10T21:15:40.976955+0000 b8699 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49897180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002955:[<0x20000002873/picture-71.jpg v939>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002955:[<0x20000002873/picture-71.jpg v939>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.955 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.699+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002876 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml auth v975 ap=1 s=7744 n(v0 rc2024-05-10T21:15:38.684011+0000 b7744 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49896100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002876:[<0x20000002873/article-60.xml v892>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002876:[<0x20000002873/article-60.xml v975>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.699+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000296d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg auth v937 ap=1 s=60035 n(v0 rc2024-05-10T21:15:41.074952+0000 b60035 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49d60000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000296d:[<0x20000002873/picture-63.jpg v937>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000296d:[<0x20000002873/picture-63.jpg v937>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028db [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml auth v965 ap=1 s=2901 n(v0 rc2024-05-10T21:15:39.012003+0000 b2901 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49895080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028db:[<0x20000002873/article-29.xml v890>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028db:[<0x20000002873/article-29.xml v965>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029d7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg auth v969 ap=1 s=39157 n(v0 rc2024-05-10T21:15:41.437943+0000 b39157 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a6eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029d7:[<0x20000002873/picture-40.jpg v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029d7:[<0x20000002873/picture-40.jpg v969>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028f3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml auth v963 ap=1 s=3964 n(v0 rc2024-05-10T21:15:39.118001+0000 b3964 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a71180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028f3:[<0x20000002873/article-22.xml v894>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028f3:[<0x20000002873/article-22.xml v963>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000289f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml auth v971 ap=1 s=3990 n(v0 rc2024-05-10T21:15:38.777009+0000 b3990 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a6e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000289f:[<0x20000002873/article-49.xml v896>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000289f:[<0x20000002873/article-49.xml v971>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a0d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg auth v994 ap=1 s=57080 n(v0 rc2024-05-10T21:15:41.587939+0000 b57080 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b5c3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a0d:[<0x20000002873/picture-23.jpg v951>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a0d:[<0x20000002873/picture-23.jpg v994>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002910 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml auth v986 ap=1 s=19742 n(v0 rc2024-05-10T21:15:39.256997+0000 b19742 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b5c3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002910:[<0x20000002873/article-14.xml v848>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002910:[<0x20000002873/article-14.xml v986>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029d2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg auth v1002 ap=1 s=10948 n(v0 rc2024-05-10T21:15:41.426943+0000 b10948 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47965700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029d2:[<0x20000002873/picture-41.jpg v953>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029d2:[<0x20000002873/picture-41.jpg v1002>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029be [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg auth v1006 ap=1 s=48305 n(v0 rc2024-05-10T21:15:41.386944+0000 b48305 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b5c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029be:[<0x20000002873/picture-45.jpg v963>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029be:[<0x20000002873/picture-45.jpg v1006>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002976 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg auth v1022 ap=1 s=56993 n(v0 rc2024-05-10T21:15:41.117951+0000 b56993 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47964100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002976:[<0x20000002873/picture-60.jpg v955>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002976:[<0x20000002873/picture-60.jpg v1022>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.703+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028a8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml auth v1008 ap=1 s=21615 n(v0 rc2024-05-10T21:15:38.814008+0000 b21615 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff490eec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028a8:[<0x20000002873/article-46.xml v850>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028a8:[<0x20000002873/article-46.xml v1008>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.956 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.703+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002964 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg auth v945 ap=1 s=59516 n(v0 rc2024-05-10T21:15:41.033953+0000 b59516 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff490ed080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002964:[<0x20000002873/picture-66.jpg v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002964:[<0x20000002873/picture-66.jpg v945>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.703+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028a4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml auth v1012 ap=1 s=29229 n(v0 rc2024-05-10T21:15:38.802008+0000 b29229 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff490ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028a4:[<0x20000002873/article-47.xml v852>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028a4:[<0x20000002873/article-47.xml v1012>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.703+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a10 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg auth v992 ap=1 s=10357 n(v0 rc2024-05-10T21:15:41.593939+0000 b10357 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47efb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a10:[<0x20000002873/picture-22.jpg v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a10:[<0x20000002873/picture-22.jpg v992>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.704+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000291c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml auth v982 ap=1 s=3867 n(v0 rc2024-05-10T21:15:39.319996+0000 b3867 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47efab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000291c:[<0x20000002873/article-11.xml v854>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000291c:[<0x20000002873/article-11.xml v982>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.704+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000288c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml auth v1016 ap=1 s=7100 n(v0 rc2024-05-10T21:15:38.729010+0000 b7100 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47efd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000288c:[<0x20000002873/article-54.xml v858>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000288c:[<0x20000002873/article-54.xml v1016>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028e6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml auth v998 ap=1 s=14535 n(v0 rc2024-05-10T21:15:39.050002+0000 b14535 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49c41b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028e6:[<0x20000002873/article-26.xml v918>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028e6:[<0x20000002873/article-26.xml v998>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002908 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml auth v990 ap=1 s=2441 n(v0 rc2024-05-10T21:15:39.210998+0000 b2441 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47efcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002908:[<0x20000002873/article-16.xml v856>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002908:[<0x20000002873/article-16.xml v990>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a25 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg auth v988 ap=1 s=65501 n(v0 rc2024-05-10T21:15:41.634938+0000 b65501 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49c40b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a25:[<0x20000002873/picture-15.jpg v965>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a25:[<0x20000002873/picture-15.jpg v988>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002982 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg auth v1018 ap=1 s=25859 n(v0 rc2024-05-10T21:15:41.185949+0000 b25859 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49c42c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002982:[<0x20000002873/picture-56.jpg v957>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002982:[<0x20000002873/picture-56.jpg v1018>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.706+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000168d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml auth v977 ap=1 s=15982 n(v0 rc2024-05-10T21:16:16.681076+0000 b15982 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49c43700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000168d:[<0x20000002873/article-10.xml v972>,<0x10000000399/blog-84 v121309>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x1000000168d:[<0x20000002873/article-10.xml v977>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.707+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029b4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg auth v1010 ap=1 s=34921 n(v0 rc2024-05-10T21:15:41.365945+0000 b34921 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49c40000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029b4:[<0x20000002873/picture-47.jpg v959>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029b4:[<0x20000002873/picture-47.jpg v1010>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.707+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a2e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg auth v984 ap=1 s=24264 n(v0 rc2024-05-10T21:15:41.652938+0000 b24264 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49397180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a2e:[<0x20000002873/picture-12.jpg v967>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a2e:[<0x20000002873/picture-12.jpg v984>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.957 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.707+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029c3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg auth v1004 ap=1 s=50233 n(v0 rc2024-05-10T21:15:41.397944+0000 b50233 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49395b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029c3:[<0x20000002873/picture-44.jpg v961>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029c3:[<0x20000002873/picture-44.jpg v1004>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.958 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.707+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028a1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml auth v1014 ap=1 s=20014 n(v0 rc2024-05-10T21:15:38.791009+0000 b20014 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49397700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028a1:[<0x20000002873/article-48.xml v922>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028a1:[<0x20000002873/article-48.xml v1014>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.958 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.708+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002937 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml auth v980 ap=1 s=31113 n(v0 rc2024-05-10T21:15:40.825958+0000 b31113 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49394b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002937:[<0x20000002873/article-4.xml v941>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002937:[<0x20000002873/article-4.xml v980>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.959 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.709+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028e9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml auth v996 ap=1 s=28618 n(v0 rc2024-05-10T21:15:39.065002+0000 b28618 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4805a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028e9:[<0x20000002873/article-25.xml v924>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028e9:[<0x20000002873/article-25.xml v996>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.959 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.709+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002952 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg auth v947 ap=1 s=41792 n(v0 rc2024-05-10T21:15:40.964955+0000 b41792 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4805a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002952:[<0x20000002873/picture-72.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002952:[<0x20000002873/picture-72.jpg v947>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.959 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.710+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000294f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg auth v939 ap=1 s=15252 n(v0 rc2024-05-10T21:15:40.953955+0000 b15252 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48058000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000294f:[<0x20000002873/picture-73.jpg v939>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000294f:[<0x20000002873/picture-73.jpg v939>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.959 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.710+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028c8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml auth v971 ap=1 s=9427 n(v0 rc2024-05-10T21:15:38.937005+0000 b9427 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff494eb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028c8:[<0x20000002873/article-35.xml v832>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028c8:[<0x20000002873/article-35.xml v971>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.959 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.711+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002979 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg auth v1020 ap=1 s=26649 n(v0 rc2024-05-10T21:15:41.130951+0000 b26649 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48059b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002979:[<0x20000002873/picture-59.jpg v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002979:[<0x20000002873/picture-59.jpg v1020>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.711+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028c0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml auth v1000 ap=1 s=21557 n(v0 rc2024-05-10T21:15:38.905006+0000 b21557 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48059600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028c0:[<0x20000002873/article-38.xml v830>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028c0:[<0x20000002873/article-38.xml v1000>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.712+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a13 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg auth v967 ap=1 s=59911 n(v0 rc2024-05-10T21:15:41.599939+0000 b59911 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48059080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a13:[<0x20000002873/picture-21.jpg v945>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a13:[<0x20000002873/picture-21.jpg v967>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.712+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002949 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg auth v949 ap=1 s=46027 n(v0 rc2024-05-10T21:15:40.929956+0000 b46027 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff494ec680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002949:[<0x20000002873/picture-75.jpg v949>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002949:[<0x20000002873/picture-75.jpg v949>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.712+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029f8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg auth v969 ap=1 s=48431 n(v0 rc2024-05-10T21:15:41.536941+0000 b48431 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497b0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029f8:[<0x20000002873/picture-30.jpg v947>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029f8:[<0x20000002873/picture-30.jpg v969>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.713+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000290c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml auth v961 ap=1 s=5903 n(v0 rc2024-05-10T21:15:39.232998+0000 b5903 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497b0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000290c:[<0x20000002873/article-15.xml v836>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000290c:[<0x20000002873/article-15.xml v961>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.713+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028bb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml auth v977 ap=1 s=20942 n(v0 rc2024-05-10T21:15:38.881006+0000 b20942 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a91080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028bb:[<0x20000002873/article-40.xml v834>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028bb:[<0x20000002873/article-40.xml v977>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.713+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029e6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg auth v973 ap=1 s=54487 n(v0 rc2024-05-10T21:15:41.478942+0000 b54487 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497af080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029e6:[<0x20000002873/picture-36.jpg v953>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029e6:[<0x20000002873/picture-36.jpg v973>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.714+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029c8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg auth v981 ap=1 s=53027 n(v0 rc2024-05-10T21:15:41.407944+0000 b53027 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497ae000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029c8:[<0x20000002873/picture-43.jpg v951>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029c8:[<0x20000002873/picture-43.jpg v981>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.715+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a16 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg auth v965 ap=1 s=53646 n(v0 rc2024-05-10T21:15:41.605939+0000 b53646 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485b0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a16:[<0x20000002873/picture-20.jpg v943>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a16:[<0x20000002873/picture-20.jpg v965>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.960 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.715+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028b4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml auth v979 ap=1 s=3151 n(v0 rc2024-05-10T21:15:38.863007+0000 b3151 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485b0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028b4:[<0x20000002873/article-42.xml v866>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028b4:[<0x20000002873/article-42.xml v979>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.715+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029e0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg auth v975 ap=1 s=25918 n(v0 rc2024-05-10T21:15:41.458943+0000 b25918 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485af080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029e0:[<0x20000002873/picture-38.jpg v941>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029e0:[<0x20000002873/picture-38.jpg v975>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.716+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028fe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml auth v963 ap=1 s=8774 n(v0 rc2024-05-10T21:15:39.165999+0000 b8774 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485af600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028fe:[<0x20000002873/article-19.xml v864>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028fe:[<0x20000002873/article-19.xml v963>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.716+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028c3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml auth v984 ap=1 s=10391 n(v0 rc2024-05-10T21:15:38.917006+0000 b10391 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485aeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028c3:[<0x20000002873/article-37.xml v868>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028c3:[<0x20000002873/article-37.xml v984>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.716+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a01 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg auth v976 ap=1 s=50410 n(v0 rc2024-05-10T21:15:41.564940+0000 b50410 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485b1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a01:[<0x20000002873/picture-27.jpg v944>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a01:[<0x20000002873/picture-27.jpg v976>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.716+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028be [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml auth v986 ap=1 s=22779 n(v0 rc2024-05-10T21:15:38.893006+0000 b22779 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4961cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028be:[<0x20000002873/article-39.xml v870>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028be:[<0x20000002873/article-39.xml v986>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028c7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml auth v982 ap=1 s=18533 n(v0 rc2024-05-10T21:15:38.930005+0000 b18533 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4961c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028c7:[<0x20000002873/article-36.xml v872>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028c7:[<0x20000002873/article-36.xml v982>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.961 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002918 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml auth v968 ap=1 s=21156 n(v0 rc2024-05-10T21:15:39.294996+0000 b21156 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ba2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002918:[<0x20000002873/article-12.xml v874>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002918:[<0x20000002873/article-12.xml v968>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002940 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml auth v966 ap=1 s=25285 n(v0 rc2024-05-10T21:15:40.882957+0000 b25285 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ba1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002940:[<0x20000002873/article-1.xml v942>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002940:[<0x20000002873/article-1.xml v966>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000299a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg auth v990 ap=1 s=62616 n(v0 rc2024-05-10T21:15:41.287947+0000 b62616 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ba2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000299a:[<0x20000002873/picture-52.jpg v956>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000299a:[<0x20000002873/picture-52.jpg v990>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.718+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029ec [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg auth v980 ap=1 s=51151 n(v0 rc2024-05-10T21:15:41.498942+0000 b51151 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ba0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029ec:[<0x20000002873/picture-34.jpg v952>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029ec:[<0x20000002873/picture-34.jpg v980>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.719+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002a22 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg auth v970 ap=1 s=9725 n(v0 rc2024-05-10T21:15:41.628938+0000 b9725 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4867b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002a22:[<0x20000002873/picture-16.jpg v950>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002a22:[<0x20000002873/picture-16.jpg v970>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.719+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002961 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg auth v948 ap=1 s=41586 n(v0 rc2024-05-10T21:15:41.019954+0000 b41586 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48679080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002961:[<0x20000002873/picture-67.jpg v948>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002961:[<0x20000002873/picture-67.jpg v948>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.719+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000294c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg auth v940 ap=1 s=12610 n(v0 rc2024-05-10T21:15:40.942955+0000 b12610 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4790d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000294c:[<0x20000002873/picture-74.jpg v940>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x2000000294c:[<0x20000002873/picture-74.jpg v940>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.720+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002993 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg auth v992 ap=1 s=35773 n(v0 rc2024-05-10T21:15:41.274947+0000 b35773 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ba1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002993:[<0x20000002873/picture-53.jpg v958>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002993:[<0x20000002873/picture-53.jpg v992>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.721+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029f2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg auth v978 ap=1 s=17927 n(v0 rc2024-05-10T21:15:41.517941+0000 b17927 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff483d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029f2:[<0x20000002873/picture-32.jpg v946>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029f2:[<0x20000002873/picture-32.jpg v978>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.722+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002946 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg auth v938 ap=1 s=29729 n(v0 rc2024-05-10T21:15:40.917956+0000 b29729 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff483d0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002946:[<0x20000002873/picture-76.jpg v938>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002946:[<0x20000002873/picture-76.jpg v938>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.964 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.722+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000028f0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml auth v974 ap=1 s=23455 n(v0 rc2024-05-10T21:15:39.101001+0000 b23455 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4790bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000028f0:[<0x20000002873/article-23.xml v860>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000028f0:[<0x20000002873/article-23.xml v974>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.723+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002905 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml auth v972 ap=1 s=10786 n(v0 rc2024-05-10T21:15:39.195999+0000 b10786 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4790ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002905:[<0x20000002873/article-17.xml v862>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002905:[<0x20000002873/article-17.xml v972>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.724+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000029af [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg auth v988 ap=1 s=35909 n(v0 rc2024-05-10T21:15:41.355945+0000 b35909 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff483ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000029af:[<0x20000002873/picture-48.jpg v954>,<0x10000000399/blog-84 v114648>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x200000029af:[<0x20000002873/picture-48.jpg v988>,<0x10000000399/blog-84 v130062>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.763+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d86 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml auth v944 ap=1 s=22116 n(v0 rc2024-05-10T21:15:35.483091+0000 b22116 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485c7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d86:[<0x20000001d11/article-19.xml v936>,<0x10000000399/blog-56 v88045>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d86:[<0x20000001d11/article-19.xml v944>,<0x10000000399/blog-56 v129606>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.764+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d85 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg auth v940 ap=1 s=48465 n(v0 rc2024-05-10T21:15:35.475091+0000 b48465 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48fb4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d85:[<0x20000001d11/picture-30.jpg v940>,<0x10000000399/blog-56 v88045>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d85:[<0x20000001d11/picture-30.jpg v940>,<0x10000000399/blog-56 v129606>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.784+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022a6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg auth v1194 ap=1 s=20925 n(v0 rc2024-05-10T21:13:41.518924+0000 b20925 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022a6:[<0x2000000218d/picture-64.jpg v1194>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022a6:[<0x2000000218d/picture-64.jpg v1194>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.784+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000219f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml auth v1196 ap=1 s=11658 n(v0 rc2024-05-10T21:13:39.480975+0000 b11658 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000219f:[<0x2000000218d/article-80.xml v1196>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000219f:[<0x2000000218d/article-80.xml v1196>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.784+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000238e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg auth v1515 ap=1 s=44740 n(v0 rc2024-05-10T21:15:18.921501+0000 b44740 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a6d6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000238e:[<0x2000000218d/picture-10.jpg v1505>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000238e:[<0x2000000218d/picture-10.jpg v1515>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.785+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023b2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg auth v1507 ap=1 s=17364 n(v0 rc2024-05-10T21:15:19.192494+0000 b17364 1=1+0) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55ff4b5e0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023b2:[<0x2000000218d/picture-1.jpg v1507>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023b2:[<0x2000000218d/picture-1.jpg v1507>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.785+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002275 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml auth v1198 ap=1 s=2777 n(v0 rc2024-05-10T21:13:41.256931+0000 b2777 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002275:[<0x2000000218d/article-9.xml v1198>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002275:[<0x2000000218d/article-9.xml v1198>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.785+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021cf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml auth v1202 ap=1 s=20353 n(v0 rc2024-05-10T21:13:40.179958+0000 b20353 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021cf:[<0x2000000218d/article-65.xml v1202>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021cf:[<0x2000000218d/article-65.xml v1202>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.785+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000220f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml auth v1200 ap=1 s=10164 n(v0 rc2024-05-10T21:13:40.838941+0000 b10164 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000220f:[<0x2000000218d/article-44.xml v1200>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000220f:[<0x2000000218d/article-44.xml v1200>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.786+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023a3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg auth v1513 ap=1 s=61880 n(v0 rc2024-05-10T21:15:19.117496+0000 b61880 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a6d8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023a3:[<0x2000000218d/picture-6.jpg v1501>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023a3:[<0x2000000218d/picture-6.jpg v1513>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.965 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.786+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000224a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml auth v1206 ap=1 s=2265 n(v0 rc2024-05-10T21:13:41.085935+0000 b2265 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000224a:[<0x2000000218d/article-24.xml v1206>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000224a:[<0x2000000218d/article-24.xml v1206>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.786+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000234d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg auth v1434 ap=1 s=38505 n(v0 rc2024-05-10T21:13:43.841866+0000 b38505 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000234d:[<0x2000000218d/picture-28.jpg v1434>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000234d:[<0x2000000218d/picture-28.jpg v1434>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.786+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021d5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml auth v1204 ap=1 s=2807 n(v0 rc2024-05-10T21:13:40.367953+0000 b2807 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a6d8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021d5:[<0x2000000218d/article-62.xml v1204>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021d5:[<0x2000000218d/article-62.xml v1204>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.786+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002195 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml auth v1208 ap=1 s=14939 n(v0 rc2024-05-10T21:13:39.360978+0000 b14939 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002195:[<0x2000000218d/article-85.xml v1208>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002195:[<0x2000000218d/article-85.xml v1208>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.787+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000218e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml auth v1210 ap=1 s=22288 n(v0 rc2024-05-10T21:13:39.338979+0000 b22288 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000218e:[<0x2000000218d/article-88.xml v1210>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000218e:[<0x2000000218d/article-88.xml v1210>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.787+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021ca [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml auth v1212 ap=1 s=4986 n(v0 rc2024-05-10T21:13:40.152958+0000 b4986 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021ca:[<0x2000000218d/article-67.xml v1212>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021ca:[<0x2000000218d/article-67.xml v1212>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.787+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002332 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg auth v1452 ap=1 s=28190 n(v0 rc2024-05-10T21:13:43.601872+0000 b28190 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002332:[<0x2000000218d/picture-37.jpg v1452>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002332:[<0x2000000218d/picture-37.jpg v1452>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.787+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002368 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg auth v1412 ap=1 s=48467 n(v0 rc2024-05-10T21:13:44.122859+0000 b48467 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002368:[<0x2000000218d/picture-17.jpg v1412>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002368:[<0x2000000218d/picture-17.jpg v1412>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.788+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002282 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml auth v1214 ap=1 s=29010 n(v0 rc2024-05-10T21:13:41.322929+0000 b29010 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002282:[<0x2000000218d/article-3.xml v1214>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002282:[<0x2000000218d/article-3.xml v1214>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.788+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002358 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg auth v1426 ap=1 s=9262 n(v0 rc2024-05-10T21:13:43.941864+0000 b9262 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002358:[<0x2000000218d/picture-24.jpg v1426>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002358:[<0x2000000218d/picture-24.jpg v1426>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.788+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000233e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg auth v1444 ap=1 s=8204 n(v0 rc2024-05-10T21:13:43.701870+0000 b8204 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b93b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000233e:[<0x2000000218d/picture-33.jpg v1444>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000233e:[<0x2000000218d/picture-33.jpg v1444>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.966 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.788+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022d8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg auth v1216 ap=1 s=14568 n(v0 rc2024-05-10T21:13:41.778918+0000 b14568 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b0f9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022d8:[<0x2000000218d/picture-57.jpg v1216>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022d8:[<0x2000000218d/picture-57.jpg v1216>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.788+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000227a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml auth v1218 ap=1 s=28609 n(v0 rc2024-05-10T21:13:41.280930+0000 b28609 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b0fb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000227a:[<0x2000000218d/article-7.xml v1218>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000227a:[<0x2000000218d/article-7.xml v1218>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.789+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002290 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg auth v1222 ap=1 s=41861 n(v0 rc2024-05-10T21:13:41.395927+0000 b41861 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b0fa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002290:[<0x2000000218d/picture-68.jpg v1222>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002290:[<0x2000000218d/picture-68.jpg v1222>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.790+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002323 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg auth v1220 ap=1 s=21253 n(v0 rc2024-05-10T21:13:42.126909+0000 b21253 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b0f9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002323:[<0x2000000218d/picture-42.jpg v1220>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002323:[<0x2000000218d/picture-42.jpg v1220>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.790+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000232c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg auth v1456 ap=1 s=61470 n(v0 rc2024-05-10T21:13:43.211882+0000 b61470 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4b0f8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000232c:[<0x2000000218d/picture-39.jpg v1456>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000232c:[<0x2000000218d/picture-39.jpg v1456>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.806+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002191 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml auth v1366 ap=1 s=31002 n(v0 rc2024-05-10T21:13:39.345978+0000 b31002 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff483b6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002191:[<0x2000000218d/article-87.xml v1366>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002191:[<0x2000000218d/article-87.xml v1366>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.806+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021d0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml auth v1364 ap=1 s=6617 n(v0 rc2024-05-10T21:13:40.197957+0000 b6617 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48026000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021d0:[<0x2000000218d/article-64.xml v1364>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021d0:[<0x2000000218d/article-64.xml v1364>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.806+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021ab [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml auth v1368 ap=1 s=12535 n(v0 rc2024-05-10T21:13:39.701969+0000 b12535 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff483b9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021ab:[<0x2000000218d/article-75.xml v1368>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021ab:[<0x2000000218d/article-75.xml v1368>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002223 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml auth v1370 ap=1 s=26703 n(v0 rc2024-05-10T21:13:40.926939+0000 b26703 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002223:[<0x2000000218d/article-37.xml v1370>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002223:[<0x2000000218d/article-37.xml v1370>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021b4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml auth v1372 ap=1 s=24589 n(v0 rc2024-05-10T21:13:39.945963+0000 b24589 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c2f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021b4:[<0x2000000218d/article-73.xml v1372>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021b4:[<0x2000000218d/article-73.xml v1372>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000221e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml auth v1374 ap=1 s=28526 n(v0 rc2024-05-10T21:13:40.908939+0000 b28526 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4978b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000221e:[<0x2000000218d/article-39.xml v1374>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000221e:[<0x2000000218d/article-39.xml v1374>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002350 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg auth v1432 ap=1 s=27056 n(v0 rc2024-05-10T21:13:43.862866+0000 b27056 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c2f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002350:[<0x2000000218d/picture-27.jpg v1432>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002350:[<0x2000000218d/picture-27.jpg v1432>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002226 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml auth v1376 ap=1 s=12572 n(v0 rc2024-05-10T21:13:40.936939+0000 b12572 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49788000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002226:[<0x2000000218d/article-36.xml v1376>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002226:[<0x2000000218d/article-36.xml v1376>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.967 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.807+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000226b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml auth v1378 ap=1 s=20736 n(v0 rc2024-05-10T21:13:41.222932+0000 b20736 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49789600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000226b:[<0x2000000218d/article-12.xml v1378>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000226b:[<0x2000000218d/article-12.xml v1378>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.808+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002304 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg auth v1380 ap=1 s=57383 n(v0 rc2024-05-10T21:13:41.957913+0000 b57383 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49789b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002304:[<0x2000000218d/picture-52.jpg v1380>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002304:[<0x2000000218d/picture-52.jpg v1380>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.808+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000233b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg auth v1446 ap=1 s=48775 n(v0 rc2024-05-10T21:13:43.676870+0000 b48775 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a571080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000233b:[<0x2000000218d/picture-34.jpg v1446>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000233b:[<0x2000000218d/picture-34.jpg v1446>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.808+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022f7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg auth v1384 ap=1 s=39298 n(v0 rc2024-05-10T21:13:41.948913+0000 b39298 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a572c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022f7:[<0x2000000218d/picture-53.jpg v1384>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022f7:[<0x2000000218d/picture-53.jpg v1384>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.808+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002287 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml auth v1382 ap=1 s=14057 n(v0 rc2024-05-10T21:13:41.346928+0000 b14057 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a572100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002287:[<0x2000000218d/article-1.xml v1382>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002287:[<0x2000000218d/article-1.xml v1382>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.809+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000236b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg auth v1410 ap=1 s=44267 n(v0 rc2024-05-10T21:13:44.160858+0000 b44267 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48b86c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000236b:[<0x2000000218d/picture-16.jpg v1410>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000236b:[<0x2000000218d/picture-16.jpg v1410>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.968 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.809+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002294 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg auth v1386 ap=1 s=64217 n(v0 rc2024-05-10T21:13:41.431926+0000 b64217 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48b85b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002294:[<0x2000000218d/picture-67.jpg v1386>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002294:[<0x2000000218d/picture-67.jpg v1386>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.810+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021a8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml auth v1390 ap=1 s=14368 n(v0 rc2024-05-10T21:13:39.644971+0000 b14368 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c5c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021a8:[<0x2000000218d/article-77.xml v1390>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021a8:[<0x2000000218d/article-77.xml v1390>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.810+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000224d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml auth v1392 ap=1 s=22544 n(v0 rc2024-05-10T21:13:41.095935+0000 b22544 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c5d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000224d:[<0x2000000218d/article-23.xml v1392>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000224d:[<0x2000000218d/article-23.xml v1392>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.810+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021cd [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml auth v1388 ap=1 s=16829 n(v0 rc2024-05-10T21:13:40.163958+0000 b16829 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48b86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021cd:[<0x2000000218d/article-66.xml v1388>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021cd:[<0x2000000218d/article-66.xml v1388>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.810+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000225e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml auth v1394 ap=1 s=30949 n(v0 rc2024-05-10T21:13:41.166933+0000 b30949 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c5ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000225e:[<0x2000000218d/article-17.xml v1394>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000225e:[<0x2000000218d/article-17.xml v1394>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.810+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002341 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg auth v1442 ap=1 s=23375 n(v0 rc2024-05-10T21:13:43.728869+0000 b23375 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48c5d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002341:[<0x2000000218d/picture-32.jpg v1442>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002341:[<0x2000000218d/picture-32.jpg v1442>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.969 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.811+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021d2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml auth v1396 ap=1 s=17661 n(v0 rc2024-05-10T21:13:40.272955+0000 b17661 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49086c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021d2:[<0x2000000218d/article-63.xml v1396>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021d2:[<0x2000000218d/article-63.xml v1396>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.811+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002311 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg auth v1398 ap=1 s=49081 n(v0 rc2024-05-10T21:13:42.007912+0000 b49081 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49085080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002311:[<0x2000000218d/picture-48.jpg v1398>,<0x10000000399/blog-68 v92172>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002311:[<0x2000000218d/picture-48.jpg v1398>,<0x10000000399/blog-68 v130124>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.846+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002f0e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml auth v1403 ap=1 s=9385 n(v0 rc2024-05-10T21:15:49.881735+0000 b9385 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a3fc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002f0e:[<0x10000000517/comment-56.xml v1394>,<0x10000000399/blog-5 v114856>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x20000002f0e:[<0x10000000517/comment-56.xml v1403>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.895+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000039c1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml auth v1472 ap=1 s=2562 n(v0 rc2024-05-10T21:16:07.966290+0000 b2562 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4857eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000039c1:[<0x10000000517/comment-52.xml v1443>,<0x10000000399/blog-5 v121365>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x200000039c1:[<0x10000000517/comment-52.xml v1472>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.896+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c7e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml auth v1331 ap=1 s=6657 n(v0 rc2024-05-10T21:15:21.594434+0000 b6657 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48590580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c7e:[<0x10000000517/comment-66.xml v1331>,<0x10000000399/blog-5 v93209>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c7e:[<0x10000000517/comment-66.xml v1331>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.911+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d73 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml auth v1362 ap=1 s=8305 n(v0 rc2024-05-10T21:15:33.037151+0000 b8305 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49afd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d73:[<0x10000000517/comment-62.xml v1362>,<0x10000000399/blog-5 v99363>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d73:[<0x10000000517/comment-62.xml v1362>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.915+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d90 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml auth v1365 ap=1 s=7912 n(v0 rc2024-05-10T21:15:36.972054+0000 b7912 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4a381700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d90:[<0x10000000517/comment-59.xml v1365>,<0x10000000399/blog-5 v99363>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d90:[<0x10000000517/comment-59.xml v1365>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.918+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000003c67 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml auth v1493 ap=1 s=4133 n(v0 rc2024-05-10T21:16:11.962192+0000 b4133 1=1+0) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=0 freezing=0 frozen=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55ff47bca680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000003c67:[<0x10000000517/comment-50.xml v1440>,<0x10000000399/blog-5 v121365>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x20000003c67:[<0x10000000517/comment-50.xml v1493>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.970 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.918+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000003f83 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml auth v1608 ap=1 s=8412 n(v0 rc2024-05-10T21:16:17.669052+0000 b8412 1=1+0) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=0 freezing=0 frozen=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55ff4c9ac000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000003f83:[<0x10000000517/comment-48.xml v1462>,<0x10000000399/blog-5 v127765>,<0x10000000208/blogtest_in v2107>,<0x10000000201/src v1784>,<0x10000000006/blogbench-1.0 v1565>,<0x10000000005/tmp v132>,<0x100000001fb/client.0 v133>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v145>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v162>,<0x1/volumes v151>]//[]","memoryvalue":"(3)0x20000003f83:[<0x10000000517/comment-48.xml v1608>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.971 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.942+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001483 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml auth v1484 ap=1 s=5228 n(v0 rc2024-05-10T21:16:15.270111+0000 b5228 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4bd01700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001483:[<0x10000000517/comment-49.xml v1461>,<0x10000000399/blog-5 v121365>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001483:[<0x10000000517/comment-49.xml v1484>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.971 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.944+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cd2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml auth v1333 ap=1 s=7071 n(v0 rc2024-05-10T21:15:24.897353+0000 b7071 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4bcff080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cd2:[<0x10000000517/comment-65.xml v1333>,<0x10000000399/blog-5 v93209>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cd2:[<0x10000000517/comment-65.xml v1333>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:33.971 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:33.950+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000003b6c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml auth v1486 ap=1 s=5736 n(v0 rc2024-05-10T21:16:10.192235+0000 b5736 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49844580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000003b6c:[<0x10000000517/comment-51.xml v1454>,<0x10000000399/blog-5 v121365>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x20000003b6c:[<0x10000000517/comment-51.xml v1486>,<0x10000000399/blog-5 v130122>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.190 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-mon[29546]: scrub queued for path: / 2024-05-10T21:17:34.190 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:34.190 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:17:34.190 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:17:34.190 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-mon[29546]: scrub summary: active paths [/] 2024-05-10T21:17:34.195 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.119+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c94 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml auth v1258 ap=1 s=8527 n(v0 rc2024-05-10T21:15:20.106471+0000 b8527 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4923c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c94:[<0x200000019ba/comment-56.xml v1192>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c94:[<0x200000019ba/comment-56.xml v1258>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.195 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.119+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml auth v1249 ap=1 s=23749 n(v0 rc2024-05-10T21:16:11.855194+0000 b23749 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49280000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000112a:[<0x200000019ba/article-74.xml v1234>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x1000000112a:[<0x200000019ba/article-74.xml v1249>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.195 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.119+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001125 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml auth v1247 ap=1 s=4583 n(v0 rc2024-05-10T21:16:11.488203+0000 b4583 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48cf9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001125:[<0x200000019ba/comment-5.xml v1217>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001125:[<0x200000019ba/comment-5.xml v1247>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.195 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.120+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000caf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml auth v1256 ap=1 s=5774 n(v0 rc2024-05-10T21:15:20.536461+0000 b5774 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48e0c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000caf:[<0x200000019ba/comment-29.xml v1198>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000caf:[<0x200000019ba/comment-29.xml v1256>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.121+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c6e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml auth v1260 ap=1 s=2675 n(v0 rc2024-05-10T21:15:19.424488+0000 b2675 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4762cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c6e:[<0x200000019ba/comment-92.xml v1188>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c6e:[<0x200000019ba/comment-92.xml v1260>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.122+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cc2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml auth v1252 ap=1 s=8555 n(v0 rc2024-05-10T21:15:21.040448+0000 b8555 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a82b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cc2:[<0x200000019ba/comment-11.xml v1190>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cc2:[<0x200000019ba/comment-11.xml v1252>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.122+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml auth v1254 ap=1 s=8755 n(v0 rc2024-05-10T21:15:20.718456+0000 b8755 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a83b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb9:[<0x200000019ba/comment-20.xml v1196>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb9:[<0x200000019ba/comment-20.xml v1254>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.123+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml auth v1233 ap=1 s=10077 n(v0 rc2024-05-10T21:15:20.372465+0000 b10077 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a85180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca7:[<0x200000019ba/comment-37.xml v1199>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca7:[<0x200000019ba/comment-37.xml v1233>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.123+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c85 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml auth v1241 ap=1 s=3486 n(v0 rc2024-05-10T21:15:19.853478+0000 b3486 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a83080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c85:[<0x200000019ba/comment-71.xml v1193>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c85:[<0x200000019ba/comment-71.xml v1241>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.123+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbd [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml auth v1223 ap=1 s=6045 n(v0 rc2024-05-10T21:15:20.846453+0000 b6045 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48986680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cbd:[<0x200000019ba/comment-16.xml v1191>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cbd:[<0x200000019ba/comment-16.xml v1223>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.124+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c92 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml auth v1239 ap=1 s=3012 n(v0 rc2024-05-10T21:15:20.087472+0000 b3012 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48747080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c92:[<0x200000019ba/comment-58.xml v1195>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c92:[<0x200000019ba/comment-58.xml v1239>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.124+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c95 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml auth v1237 ap=1 s=6374 n(v0 rc2024-05-10T21:15:20.122471+0000 b6374 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48748100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c95:[<0x200000019ba/comment-55.xml v1185>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c95:[<0x200000019ba/comment-55.xml v1237>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.125+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000caa [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml auth v1231 ap=1 s=3622 n(v0 rc2024-05-10T21:15:20.461463+0000 b3622 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff495c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000caa:[<0x200000019ba/comment-34.xml v1205>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000caa:[<0x200000019ba/comment-34.xml v1231>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.125+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml auth v1229 ap=1 s=2832 n(v0 rc2024-05-10T21:15:20.602459+0000 b2832 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff495c3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb4:[<0x200000019ba/comment-25.xml v1197>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb4:[<0x200000019ba/comment-25.xml v1229>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.125+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml auth v1235 ap=1 s=3335 n(v0 rc2024-05-10T21:15:20.305466+0000 b3335 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49e11180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca3:[<0x200000019ba/comment-41.xml v1187>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca3:[<0x200000019ba/comment-41.xml v1235>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.196 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.126+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c81 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml auth v1243 ap=1 s=8710 n(v0 rc2024-05-10T21:15:19.799479+0000 b8710 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4895cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c81:[<0x200000019ba/comment-75.xml v1201>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c81:[<0x200000019ba/comment-75.xml v1243>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.127+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml auth v1225 ap=1 s=4985 n(v0 rc2024-05-10T21:15:20.644458+0000 b4985 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48b27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb6:[<0x200000019ba/comment-23.xml v1189>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb6:[<0x200000019ba/comment-23.xml v1225>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.128+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml auth v1227 ap=1 s=5115 n(v0 rc2024-05-10T21:15:20.623459+0000 b5115 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff477d3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb5:[<0x200000019ba/comment-24.xml v1203>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb5:[<0x200000019ba/comment-24.xml v1227>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.128+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml auth v1245 ap=1 s=9548 n(v0 rc2024-05-10T21:15:20.215469+0000 b9548 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b5e0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9d:[<0x200000019ba/comment-47.xml v1187>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9d:[<0x200000019ba/comment-47.xml v1245>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.128+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml auth v1243 ap=1 s=6843 n(v0 rc2024-05-10T21:15:20.229468+0000 b6843 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cd6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9e:[<0x200000019ba/comment-46.xml v1191>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9e:[<0x200000019ba/comment-46.xml v1243>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.128+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c86 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml auth v1253 ap=1 s=2785 n(v0 rc2024-05-10T21:15:19.860477+0000 b2785 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cd4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c86:[<0x200000019ba/comment-70.xml v1183>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c86:[<0x200000019ba/comment-70.xml v1253>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.131+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c77 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml auth v1259 ap=1 s=8082 n(v0 rc2024-05-10T21:15:19.615483+0000 b8082 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b4ce100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c77:[<0x200000019ba/comment-83.xml v1207>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c77:[<0x200000019ba/comment-83.xml v1259>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.131+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c87 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml auth v1251 ap=1 s=8986 n(v0 rc2024-05-10T21:15:19.869477+0000 b8986 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b4cf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c87:[<0x200000019ba/comment-69.xml v1195>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c87:[<0x200000019ba/comment-69.xml v1251>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.132+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c71 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml auth v1261 ap=1 s=7502 n(v0 rc2024-05-10T21:15:19.486487+0000 b7502 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47b5d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c71:[<0x200000019ba/comment-89.xml v1197>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c71:[<0x200000019ba/comment-89.xml v1261>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.197 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.133+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c82 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml auth v1257 ap=1 s=3224 n(v0 rc2024-05-10T21:15:19.818478+0000 b3224 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49395600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c82:[<0x200000019ba/comment-74.xml v1199>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c82:[<0x200000019ba/comment-74.xml v1257>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.133+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c70 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml auth v1263 ap=1 s=7679 n(v0 rc2024-05-10T21:15:19.467487+0000 b7679 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49394580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c70:[<0x200000019ba/comment-90.xml v1189>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c70:[<0x200000019ba/comment-90.xml v1263>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.134+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml auth v1241 ap=1 s=6952 n(v0 rc2024-05-10T21:15:20.385464+0000 b6952 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47c8d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca8:[<0x200000019ba/comment-36.xml v1201>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca8:[<0x200000019ba/comment-36.xml v1241>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.135+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml auth v1247 ap=1 s=8615 n(v0 rc2024-05-10T21:15:20.001474+0000 b8615 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47752100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8c:[<0x200000019ba/comment-64.xml v1209>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8c:[<0x200000019ba/comment-64.xml v1247>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.135+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c84 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml auth v1255 ap=1 s=7362 n(v0 rc2024-05-10T21:15:19.846478+0000 b7362 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47750b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c84:[<0x200000019ba/comment-72.xml v1193>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c84:[<0x200000019ba/comment-72.xml v1255>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.135+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001126 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml auth v1234 ap=1 s=3012 n(v0 rc2024-05-10T21:16:11.500203+0000 b3012 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff47743b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001126:[<0x200000019ba/comment-4.xml v1227>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001126:[<0x200000019ba/comment-4.xml v1234>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.136+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml auth v1237 ap=1 s=7382 n(v0 rc2024-05-10T21:15:20.910451+0000 b7382 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a46b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cbf:[<0x200000019ba/comment-14.xml v1205>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cbf:[<0x200000019ba/comment-14.xml v1237>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.137+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c89 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml auth v1249 ap=1 s=9738 n(v0 rc2024-05-10T21:15:19.903476+0000 b9738 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a49180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c89:[<0x200000019ba/comment-67.xml v1185>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c89:[<0x200000019ba/comment-67.xml v1249>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.139+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cba [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml auth v1239 ap=1 s=2053 n(v0 rc2024-05-10T21:15:20.738456+0000 b2053 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4bea1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cba:[<0x200000019ba/comment-19.xml v1203>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cba:[<0x200000019ba/comment-19.xml v1239>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.140+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml auth v1256 ap=1 s=4824 n(v0 rc2024-05-10T21:15:20.668457+0000 b4824 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4be9f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb7:[<0x200000019ba/comment-22.xml v1213>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb7:[<0x200000019ba/comment-22.xml v1256>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.140+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001128 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml auth v1245 ap=1 s=7198 n(v0 rc2024-05-10T21:16:11.522203+0000 b7198 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4be9f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001128:[<0x200000019ba/comment-2.xml v1230>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001128:[<0x200000019ba/comment-2.xml v1245>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.140+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml auth v1262 ap=1 s=9105 n(v0 rc2024-05-10T21:15:20.261467+0000 b9105 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4bea0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca0:[<0x200000019ba/comment-44.xml v1203>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca0:[<0x200000019ba/comment-44.xml v1262>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.198 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.141+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml auth v1258 ap=1 s=9316 n(v0 rc2024-05-10T21:15:20.563460+0000 b9316 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49756000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb2:[<0x200000019ba/comment-27.xml v1207>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb2:[<0x200000019ba/comment-27.xml v1258>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.141+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c80 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml auth v1270 ap=1 s=7853 n(v0 rc2024-05-10T21:15:19.788479+0000 b7853 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48bbe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c80:[<0x200000019ba/comment-76.xml v1189>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c80:[<0x200000019ba/comment-76.xml v1270>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.142+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c98 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml auth v1266 ap=1 s=7763 n(v0 rc2024-05-10T21:15:20.166470+0000 b7763 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48104c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c98:[<0x200000019ba/comment-52.xml v1191>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c98:[<0x200000019ba/comment-52.xml v1266>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.144+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml auth v1268 ap=1 s=9425 n(v0 rc2024-05-10T21:15:19.927476+0000 b9425 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48102580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8a:[<0x200000019ba/comment-66.xml v1211>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8a:[<0x200000019ba/comment-66.xml v1268>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.144+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001123 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml auth v1247 ap=1 s=4285 n(v0 rc2024-05-10T21:16:11.457204+0000 b4285 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48103b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001123:[<0x200000019ba/comment-7.xml v1223>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001123:[<0x200000019ba/comment-7.xml v1247>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.145+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml auth v1252 ap=1 s=5001 n(v0 rc2024-05-10T21:15:20.900452+0000 b5001 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48679600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cbe:[<0x200000019ba/comment-15.xml v1199>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cbe:[<0x200000019ba/comment-15.xml v1252>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.146+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml auth v1260 ap=1 s=10010 n(v0 rc2024-05-10T21:15:20.273467+0000 b10010 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47c87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca1:[<0x200000019ba/comment-43.xml v1193>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca1:[<0x200000019ba/comment-43.xml v1260>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.146+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbc [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml auth v1254 ap=1 s=9967 n(v0 rc2024-05-10T21:15:20.793454+0000 b9967 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47c87180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cbc:[<0x200000019ba/comment-17.xml v1197>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cbc:[<0x200000019ba/comment-17.xml v1254>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.147+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001121 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml auth v1249 ap=1 s=7579 n(v0 rc2024-05-10T21:16:11.436205+0000 b7579 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff47c86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001121:[<0x200000019ba/comment-9.xml v1220>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001121:[<0x200000019ba/comment-9.xml v1249>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.199 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.147+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml auth v1264 ap=1 s=7238 n(v0 rc2024-05-10T21:15:20.246468+0000 b7238 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47c85080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9f:[<0x200000019ba/comment-45.xml v1195>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9f:[<0x200000019ba/comment-45.xml v1264>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.148+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c76 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml auth v1272 ap=1 s=9458 n(v0 rc2024-05-10T21:15:19.609484+0000 b9458 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47c84b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c76:[<0x200000019ba/comment-84.xml v1187>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c76:[<0x200000019ba/comment-84.xml v1272>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.148+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c79 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml auth v1276 ap=1 s=6633 n(v0 rc2024-05-10T21:15:19.660482+0000 b6633 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4998a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c79:[<0x200000019ba/comment-81.xml v1194>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c79:[<0x200000019ba/comment-81.xml v1276>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.148+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001120 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml auth v1251 ap=1 s=5054 n(v0 rc2024-05-10T21:16:11.428205+0000 b5054 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4998cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001120:[<0x200000019ba/comment-10.xml v1209>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001120:[<0x200000019ba/comment-10.xml v1251>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.149+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c97 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml auth v1266 ap=1 s=2826 n(v0 rc2024-05-10T21:15:20.149470+0000 b2826 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4998ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c97:[<0x200000019ba/comment-53.xml v1182>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c97:[<0x200000019ba/comment-53.xml v1266>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.149+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001124 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml auth v1257 ap=1 s=6195 n(v0 rc2024-05-10T21:16:11.474204+0000 b6195 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4998b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001124:[<0x200000019ba/comment-6.xml v1223>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001124:[<0x200000019ba/comment-6.xml v1257>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.151+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c75 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml auth v1278 ap=1 s=8803 n(v0 rc2024-05-10T21:15:19.597484+0000 b8803 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff492ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c75:[<0x200000019ba/comment-85.xml v1192>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c75:[<0x200000019ba/comment-85.xml v1278>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.151+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml auth v1262 ap=1 s=3520 n(v0 rc2024-05-10T21:15:20.415464+0000 b3520 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4bf4fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca9:[<0x200000019ba/comment-35.xml v1202>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca9:[<0x200000019ba/comment-35.xml v1262>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.152+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml auth v1260 ap=1 s=9165 n(v0 rc2024-05-10T21:15:20.545460+0000 b9165 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4bf4f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb1:[<0x200000019ba/comment-28.xml v1188>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb1:[<0x200000019ba/comment-28.xml v1260>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.152+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c93 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml auth v1268 ap=1 s=8975 n(v0 rc2024-05-10T21:15:20.098472+0000 b8975 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4bf50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c93:[<0x200000019ba/comment-57.xml v1184>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c93:[<0x200000019ba/comment-57.xml v1268>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.200 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.154+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml auth v1270 ap=1 s=5881 n(v0 rc2024-05-10T21:15:20.051473+0000 b5881 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff473f8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8f:[<0x200000019ba/comment-61.xml v1196>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8f:[<0x200000019ba/comment-61.xml v1270>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.154+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml auth v1264 ap=1 s=6563 n(v0 rc2024-05-10T21:15:20.353465+0000 b6563 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49534b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca6:[<0x200000019ba/comment-38.xml v1186>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca6:[<0x200000019ba/comment-38.xml v1264>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.155+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c88 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml auth v1272 ap=1 s=2697 n(v0 rc2024-05-10T21:15:19.887477+0000 b2697 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49535b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c88:[<0x200000019ba/comment-68.xml v1180>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c88:[<0x200000019ba/comment-68.xml v1272>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.155+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001154 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml auth v1255 ap=1 s=20140 n(v0 rc2024-05-10T21:16:12.352182+0000 b20140 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49536c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001154:[<0x200000019ba/article-49.xml v1238>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001154:[<0x200000019ba/article-49.xml v1255>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.156+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001122 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml auth v1253 ap=1 s=2485 n(v0 rc2024-05-10T21:16:11.447205+0000 b2485 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49537180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001122:[<0x200000019ba/comment-8.xml v1218>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001122:[<0x200000019ba/comment-8.xml v1253>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.157+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c7a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml auth v1274 ap=1 s=7563 n(v0 rc2024-05-10T21:15:19.671482+0000 b7563 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49534000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c7a:[<0x200000019ba/comment-80.xml v1178>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c7a:[<0x200000019ba/comment-80.xml v1274>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.159+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml auth v1185 ap=1 s=9673 n(v0 rc2024-05-10T21:15:20.026473+0000 b9673 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ad22b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8e:[<0x200000019ba/comment-62.xml v1157>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8e:[<0x200000019ba/comment-62.xml v1185>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.164+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c91 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml auth v1183 ap=1 s=4339 n(v0 rc2024-05-10T21:15:20.074472+0000 b4339 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4c3b7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c91:[<0x200000019ba/comment-59.xml v1153>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c91:[<0x200000019ba/comment-59.xml v1183>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.167+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c7d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml auth v1187 ap=1 s=7920 n(v0 rc2024-05-10T21:15:19.767480+0000 b7920 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4c3b5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c7d:[<0x200000019ba/comment-77.xml v1163>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c7d:[<0x200000019ba/comment-77.xml v1187>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.170+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml auth v1181 ap=1 s=3117 n(v0 rc2024-05-10T21:15:20.186469+0000 b3117 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4cd53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9a:[<0x200000019ba/comment-50.xml v1161>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9a:[<0x200000019ba/comment-50.xml v1181>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.201 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.172+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml auth v1179 ap=1 s=4854 n(v0 rc2024-05-10T21:15:20.204469+0000 b4854 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4cd53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9c:[<0x200000019ba/comment-48.xml v1159>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9c:[<0x200000019ba/comment-48.xml v1179>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.175+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cad [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml auth v1177 ap=1 s=3600 n(v0 rc2024-05-10T21:15:20.505461+0000 b3600 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49536680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cad:[<0x200000019ba/comment-31.xml v1155>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cad:[<0x200000019ba/comment-31.xml v1177>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.177+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c83 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml auth v1263 ap=1 s=9096 n(v0 rc2024-05-10T21:15:19.831478+0000 b9096 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b5a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c83:[<0x200000019ba/comment-73.xml v1199>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c83:[<0x200000019ba/comment-73.xml v1263>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.178+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c90 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml auth v1259 ap=1 s=3051 n(v0 rc2024-05-10T21:15:20.059472+0000 b3051 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b59eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c90:[<0x200000019ba/comment-60.xml v1203>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c90:[<0x200000019ba/comment-60.xml v1259>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.180+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c7c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml auth v1265 ap=1 s=4067 n(v0 rc2024-05-10T21:15:19.743480+0000 b4067 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4b59f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c7c:[<0x200000019ba/comment-78.xml v1191>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c7c:[<0x200000019ba/comment-78.xml v1265>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.180+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001129 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml auth v1246 ap=1 s=6416 n(v0 rc2024-05-10T21:16:11.538202+0000 b6416 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff4a2cd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001129:[<0x200000019ba/comment-1.xml v1239>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001129:[<0x200000019ba/comment-1.xml v1246>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.184+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c72 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml auth v1273 ap=1 s=7770 n(v0 rc2024-05-10T21:15:19.503486+0000 b7770 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2ca580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c72:[<0x200000019ba/comment-88.xml v1193>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c72:[<0x200000019ba/comment-88.xml v1273>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.186+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml auth v1255 ap=1 s=4546 n(v0 rc2024-05-10T21:15:20.316466+0000 b4546 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2cab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca4:[<0x200000019ba/comment-40.xml v1213>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca4:[<0x200000019ba/comment-40.xml v1255>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.187+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c7b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml auth v1267 ap=1 s=7185 n(v0 rc2024-05-10T21:15:19.680482+0000 b7185 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2ca000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c7b:[<0x200000019ba/comment-79.xml v1197>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c7b:[<0x200000019ba/comment-79.xml v1267>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.188+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cab [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml auth v1251 ap=1 s=10012 n(v0 rc2024-05-10T21:15:20.483462+0000 b10012 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2cd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cab:[<0x200000019ba/comment-33.xml v1217>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cab:[<0x200000019ba/comment-33.xml v1251>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.188+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml auth v1253 ap=1 s=6493 n(v0 rc2024-05-10T21:15:20.331466+0000 b6493 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49367180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca5:[<0x200000019ba/comment-39.xml v1195>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca5:[<0x200000019ba/comment-39.xml v1253>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.202 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.189+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c74 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml auth v1271 ap=1 s=5162 n(v0 rc2024-05-10T21:15:19.522486+0000 b5162 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49364b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c74:[<0x200000019ba/comment-86.xml v1201>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c74:[<0x200000019ba/comment-86.xml v1271>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.203 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.192+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cac [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml auth v1249 ap=1 s=6996 n(v0 rc2024-05-10T21:15:20.498462+0000 b6996 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a221080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cac:[<0x200000019ba/comment-32.xml v1219>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cac:[<0x200000019ba/comment-32.xml v1249>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.203 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.193+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml auth v1261 ap=1 s=3268 n(v0 rc2024-05-10T21:15:20.011474+0000 b3268 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a223180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8d:[<0x200000019ba/comment-63.xml v1207>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8d:[<0x200000019ba/comment-63.xml v1261>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.203 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.193+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c78 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml auth v1269 ap=1 s=3506 n(v0 rc2024-05-10T21:15:19.632483+0000 b3506 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a222100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c78:[<0x200000019ba/comment-82.xml v1209>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c78:[<0x200000019ba/comment-82.xml v1269>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.203 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.194+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c96 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml auth v1257 ap=1 s=4565 n(v0 rc2024-05-10T21:15:20.135471+0000 b4565 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a220580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c96:[<0x200000019ba/comment-54.xml v1205>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c96:[<0x200000019ba/comment-54.xml v1257>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.203 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.195+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c6f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml auth v1275 ap=1 s=6057 n(v0 rc2024-05-10T21:15:19.444488+0000 b6057 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a223700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c6f:[<0x200000019ba/comment-91.xml v1211>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c6f:[<0x200000019ba/comment-91.xml v1275>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:34 smithi092 ceph-mon[34415]: scrub queued for path: / 2024-05-10T21:17:34.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:34 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:34.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:34 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:17:34.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:34 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:17:34.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:34 smithi092 ceph-mon[34415]: scrub summary: active paths [/] 2024-05-10T21:17:34.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:34 smithi142 ceph-mon[34846]: scrub queued for path: / 2024-05-10T21:17:34.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:34 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:34.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:34 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:17:34.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:34 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:17:34.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:34 smithi142 ceph-mon[34846]: scrub summary: active paths [/] 2024-05-10T21:17:34.443 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.201+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cc0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml auth v1251 ap=1 s=10121 n(v0 rc2024-05-10T21:15:20.937451+0000 b10121 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a5b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cc0:[<0x200000019ba/comment-13.xml v1202>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cc0:[<0x200000019ba/comment-13.xml v1251>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.443 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.201+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml auth v1255 ap=1 s=3052 n(v0 rc2024-05-10T21:15:20.690457+0000 b3052 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a58b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb8:[<0x200000019ba/comment-21.xml v1198>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb8:[<0x200000019ba/comment-21.xml v1255>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.202+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cb3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml auth v1257 ap=1 s=2659 n(v0 rc2024-05-10T21:15:20.579460+0000 b2659 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48e74580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cb3:[<0x200000019ba/comment-26.xml v1214>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cb3:[<0x200000019ba/comment-26.xml v1257>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.210+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d79 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml auth v1271 ap=1 s=31427 n(v0 rc2024-05-10T21:15:33.725134+0000 b31427 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4801cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d79:[<0x200000019ba/article-1.xml v1231>,<0x10000000399/blog-49 v99383>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d79:[<0x200000019ba/article-1.xml v1271>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.210+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cae [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml auth v1259 ap=1 s=2221 n(v0 rc2024-05-10T21:15:20.514461+0000 b2221 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4801c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cae:[<0x200000019ba/comment-30.xml v1204>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cae:[<0x200000019ba/comment-30.xml v1259>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.213+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001127 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml auth v1246 ap=1 s=2093 n(v0 rc2024-05-10T21:16:11.511203+0000 b2093 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49225180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001127:[<0x200000019ba/comment-3.xml v1235>,<0x10000000399/blog-49 v107049>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001127:[<0x200000019ba/comment-3.xml v1246>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.214+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cbb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml auth v1253 ap=1 s=8550 n(v0 rc2024-05-10T21:15:20.785455+0000 b8550 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49224c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cbb:[<0x200000019ba/comment-18.xml v1210>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cbb:[<0x200000019ba/comment-18.xml v1253>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.218+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cc1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml auth v1249 ap=1 s=2657 n(v0 rc2024-05-10T21:15:20.955450+0000 b2657 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49247180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cc1:[<0x200000019ba/comment-12.xml v1212>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cc1:[<0x200000019ba/comment-12.xml v1249>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.219+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d78 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg auth v1273 ap=1 s=42498 n(v0 rc2024-05-10T21:15:33.718134+0000 b42498 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49246680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d78:[<0x200000019ba/picture-74.jpg v1227>,<0x10000000399/blog-49 v99383>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d78:[<0x200000019ba/picture-74.jpg v1273>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.220+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml auth v1263 ap=1 s=9477 n(v0 rc2024-05-10T21:15:20.191469+0000 b9477 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49245080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9b:[<0x200000019ba/comment-49.xml v1200>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c9b:[<0x200000019ba/comment-49.xml v1263>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.220+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c73 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml auth v1269 ap=1 s=7745 n(v0 rc2024-05-10T21:15:19.513486+0000 b7745 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a8edb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c73:[<0x200000019ba/comment-87.xml v1196>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c73:[<0x200000019ba/comment-87.xml v1269>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.221+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml auth v1261 ap=1 s=7244 n(v0 rc2024-05-10T21:15:20.289467+0000 b7244 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a8ecb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca2:[<0x200000019ba/comment-42.xml v1206>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000ca2:[<0x200000019ba/comment-42.xml v1261>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.444 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.223+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c8b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml auth v1267 ap=1 s=6989 n(v0 rc2024-05-10T21:15:19.969475+0000 b6989 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497cd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c8b:[<0x200000019ba/comment-65.xml v1194>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c8b:[<0x200000019ba/comment-65.xml v1267>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.445 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.225+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c99 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml auth v1265 ap=1 s=4830 n(v0 rc2024-05-10T21:15:20.178470+0000 b4830 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4908c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c99:[<0x200000019ba/comment-51.xml v1192>,<0x10000000399/blog-49 v92866>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000c99:[<0x200000019ba/comment-51.xml v1265>,<0x10000000399/blog-49 v129602>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:17:34.701 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.568+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023cf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml auth v919 ap=1 s=13467 n(v0 rc2024-05-10T21:15:19.312491+0000 b13467 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023cf:[<0x200000023b5/article-59.xml v664>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023cf:[<0x200000023b5/article-59.xml v919>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.701 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.569+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000246b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg auth v809 ap=1 s=62567 n(v0 rc2024-05-10T21:15:20.795454+0000 b62567 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000246b:[<0x200000023b5/picture-13.jpg v624>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000246b:[<0x200000023b5/picture-13.jpg v809>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.701 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.571+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000245a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg auth v829 ap=1 s=28452 n(v0 rc2024-05-10T21:15:20.609459+0000 b28452 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000245a:[<0x200000023b5/picture-18.jpg v680>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000245a:[<0x200000023b5/picture-18.jpg v829>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.701 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.572+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002430 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml auth v831 ap=1 s=8407 n(v0 rc2024-05-10T21:15:20.241468+0000 b8407 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002430:[<0x200000023b5/article-18.xml v710>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002430:[<0x200000023b5/article-18.xml v831>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.701 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.572+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002457 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg auth v833 ap=1 s=49906 n(v0 rc2024-05-10T21:15:20.579460+0000 b49906 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002457:[<0x200000023b5/picture-19.jpg v678>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002457:[<0x200000023b5/picture-19.jpg v833>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.572+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023e8 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml auth v903 ap=1 s=27483 n(v0 rc2024-05-10T21:15:19.428488+0000 b27483 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a14b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023e8:[<0x200000023b5/article-51.xml v578>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023e8:[<0x200000023b5/article-51.xml v903>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.573+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000241e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml auth v855 ap=1 s=25952 n(v0 rc2024-05-10T21:15:20.123471+0000 b25952 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a17180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000241e:[<0x200000023b5/article-27.xml v688>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000241e:[<0x200000023b5/article-27.xml v855>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.573+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024a4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg auth v773 ap=1 s=36152 n(v0 rc2024-05-10T21:15:21.932426+0000 b36152 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a15600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024a4:[<0x200000023b5/picture-4.jpg v722>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024a4:[<0x200000023b5/picture-4.jpg v773>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.573+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024b9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg auth v765 ap=1 s=11666 n(v0 rc2024-05-10T21:15:22.215419+0000 b11666 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a16c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024b9:[<0x200000023b5/picture-2.jpg v652>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024b9:[<0x200000023b5/picture-2.jpg v765>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.573+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000242c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml auth v839 ap=1 s=17372 n(v0 rc2024-05-10T21:15:20.218469+0000 b17372 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a14000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000242c:[<0x200000023b5/article-20.xml v690>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000242c:[<0x200000023b5/article-20.xml v839>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.574+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023c7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml auth v923 ap=1 s=24718 n(v0 rc2024-05-10T21:15:19.288492+0000 b24718 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48938680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023c7:[<0x200000023b5/article-61.xml v642>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023c7:[<0x200000023b5/article-61.xml v923>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.574+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002467 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg auth v813 ap=1 s=18095 n(v0 rc2024-05-10T21:15:20.750455+0000 b18095 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48939180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002467:[<0x200000023b5/picture-14.jpg v720>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002467:[<0x200000023b5/picture-14.jpg v813>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.574+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002402 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml auth v883 ap=1 s=11539 n(v0 rc2024-05-10T21:15:19.846478+0000 b11539 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48937600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002402:[<0x200000023b5/article-41.xml v614>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002402:[<0x200000023b5/article-41.xml v883>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.575+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023fa [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml auth v891 ap=1 s=12143 n(v0 rc2024-05-10T21:15:19.776479+0000 b12143 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48936580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023fa:[<0x200000023b5/article-45.xml v586>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023fa:[<0x200000023b5/article-45.xml v891>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.575+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000248d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg auth v785 ap=1 s=49529 n(v0 rc2024-05-10T21:15:21.580435+0000 b49529 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000248d:[<0x200000023b5/picture-7.jpg v708>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000248d:[<0x200000023b5/picture-7.jpg v785>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.702 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.575+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002483 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg auth v793 ap=1 s=29246 n(v0 rc2024-05-10T21:15:21.341441+0000 b29246 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002483:[<0x200000023b5/picture-9.jpg v672>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002483:[<0x200000023b5/picture-9.jpg v793>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.576+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000249e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg auth v797 ap=1 s=50812 n(v0 rc2024-05-10T21:15:21.866428+0000 b50812 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a3ecb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000249e:[<0x200000023b5/picture-10.jpg v682>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000249e:[<0x200000023b5/picture-10.jpg v797>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.576+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024c3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg auth v761 ap=1 s=63200 n(v0 rc2024-05-10T21:15:22.521412+0000 b63200 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a3ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024c3:[<0x200000023b5/picture-1.jpg v736>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024c3:[<0x200000023b5/picture-1.jpg v761>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.576+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002416 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml auth v863 ap=1 s=8298 n(v0 rc2024-05-10T21:15:20.065472+0000 b8298 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002416:[<0x200000023b5/article-31.xml v580>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002416:[<0x200000023b5/article-31.xml v863>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.576+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002442 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml auth v795 ap=1 s=32635 n(v0 rc2024-05-10T21:15:20.364465+0000 b32635 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a3ef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002442:[<0x200000023b5/article-9.xml v620>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002442:[<0x200000023b5/article-9.xml v795>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.577+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023fc [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml auth v889 ap=1 s=28268 n(v0 rc2024-05-10T21:15:19.793479+0000 b28268 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a3eec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023fc:[<0x200000023b5/article-44.xml v596>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023fc:[<0x200000023b5/article-44.xml v889>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.577+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002495 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg auth v781 ap=1 s=36701 n(v0 rc2024-05-10T21:15:21.689432+0000 b36701 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482f1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002495:[<0x200000023b5/picture-6.jpg v636>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002495:[<0x200000023b5/picture-6.jpg v781>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.577+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023bb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml auth v931 ap=1 s=2700 n(v0 rc2024-05-10T21:15:19.241493+0000 b2700 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47701080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023bb:[<0x200000023b5/article-65.xml v568>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023bb:[<0x200000023b5/article-65.xml v931>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.577+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023c5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml auth v925 ap=1 s=16401 n(v0 rc2024-05-10T21:15:19.276492+0000 b16401 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47703180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023c5:[<0x200000023b5/article-62.xml v574>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023c5:[<0x200000023b5/article-62.xml v925>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.577+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002424 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml auth v847 ap=1 s=25995 n(v0 rc2024-05-10T21:15:20.159470+0000 b25995 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47703700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002424:[<0x200000023b5/article-24.xml v600>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002424:[<0x200000023b5/article-24.xml v847>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.578+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002577 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg auth v757 ap=1 s=65037 n(v0 rc2024-05-10T21:15:27.937277+0000 b65037 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff47700580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002577:[<0x200000023b5/picture-92.jpg v742>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002577:[<0x200000023b5/picture-92.jpg v757>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.578+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000245e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg auth v825 ap=1 s=39490 n(v0 rc2024-05-10T21:15:20.649458+0000 b39490 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47701b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000245e:[<0x200000023b5/picture-17.jpg v670>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000245e:[<0x200000023b5/picture-17.jpg v825>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.703 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.578+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000244e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml auth v771 ap=1 s=10005 n(v0 rc2024-05-10T21:15:20.474462+0000 b10005 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47702680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000244e:[<0x200000023b5/article-3.xml v666>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000244e:[<0x200000023b5/article-3.xml v771>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.578+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002446 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml auth v787 ap=1 s=7205 n(v0 rc2024-05-10T21:15:20.399464+0000 b7205 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47701600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002446:[<0x200000023b5/article-7.xml v646>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002446:[<0x200000023b5/article-7.xml v787>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2289>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023db [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml auth v911 ap=1 s=9233 n(v0 rc2024-05-10T21:15:19.357490+0000 b9233 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f99700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023db:[<0x200000023b5/article-55.xml v706>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023db:[<0x200000023b5/article-55.xml v911>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002414 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml auth v865 ap=1 s=26387 n(v0 rc2024-05-10T21:15:20.048473+0000 b26387 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f98680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002414:[<0x200000023b5/article-32.xml v656>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002414:[<0x200000023b5/article-32.xml v865>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.680+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000242a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml auth v841 ap=1 s=24165 n(v0 rc2024-05-10T21:15:20.207469+0000 b24165 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f98100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000242a:[<0x200000023b5/article-21.xml v616>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000242a:[<0x200000023b5/article-21.xml v841>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.680+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002488 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg auth v789 ap=1 s=17564 n(v0 rc2024-05-10T21:15:21.452438+0000 b17564 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f98c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002488:[<0x200000023b5/picture-8.jpg v702>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002488:[<0x200000023b5/picture-8.jpg v789>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.681+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002444 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml auth v791 ap=1 s=12666 n(v0 rc2024-05-10T21:15:20.380465+0000 b12666 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f99180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002444:[<0x200000023b5/article-8.xml v738>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002444:[<0x200000023b5/article-8.xml v791>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.681+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000243a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml auth v811 ap=1 s=1710 n(v0 rc2024-05-10T21:15:20.310466+0000 b1710 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f97080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000243a:[<0x200000023b5/article-13.xml v648>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000243a:[<0x200000023b5/article-13.xml v811>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002418 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml auth v861 ap=1 s=21164 n(v0 rc2024-05-10T21:15:20.081472+0000 b21164 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f97b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002418:[<0x200000023b5/article-30.xml v598>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002418:[<0x200000023b5/article-30.xml v861>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002448 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml auth v783 ap=1 s=19838 n(v0 rc2024-05-10T21:15:20.422464+0000 b19838 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f96580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002448:[<0x200000023b5/article-6.xml v632>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002448:[<0x200000023b5/article-6.xml v783>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023eb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml auth v901 ap=1 s=19000 n(v0 rc2024-05-10T21:15:19.452488+0000 b19000 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f96b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023eb:[<0x200000023b5/article-50.xml v626>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023eb:[<0x200000023b5/article-50.xml v901>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.704 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002451 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml auth v767 ap=1 s=26685 n(v0 rc2024-05-10T21:15:20.494462+0000 b26685 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47f96000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002451:[<0x200000023b5/article-2.xml v726>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002451:[<0x200000023b5/article-2.xml v767>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.682+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024c7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml auth v913 ap=1 s=8111 n(v0 rc2024-05-10T21:15:22.488412+0000 b8111 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a4f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024c7:[<0x200000023b5/article-56.xml v704>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024c7:[<0x200000023b5/article-56.xml v913>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.683+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023fe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml auth v887 ap=1 s=9958 n(v0 rc2024-05-10T21:15:19.811479+0000 b9958 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a4e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023fe:[<0x200000023b5/article-43.xml v660>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023fe:[<0x200000023b5/article-43.xml v887>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.683+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002412 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml auth v867 ap=1 s=22889 n(v0 rc2024-05-10T21:15:20.034473+0000 b22889 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a4c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002412:[<0x200000023b5/article-33.xml v712>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002412:[<0x200000023b5/article-33.xml v867>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.683+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000249d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg auth v777 ap=1 s=17207 n(v0 rc2024-05-10T21:15:21.792430+0000 b17207 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a4d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000249d:[<0x200000023b5/picture-5.jpg v718>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000249d:[<0x200000023b5/picture-5.jpg v777>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.683+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023d5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml auth v915 ap=1 s=16285 n(v0 rc2024-05-10T21:15:19.331491+0000 b16285 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49a4ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023d5:[<0x200000023b5/article-57.xml v604>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023d5:[<0x200000023b5/article-57.xml v915>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.684+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023e3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml auth v907 ap=1 s=32667 n(v0 rc2024-05-10T21:15:19.381489+0000 b32667 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48393700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023e3:[<0x200000023b5/article-53.xml v570>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023e3:[<0x200000023b5/article-53.xml v907>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023d1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml auth v917 ap=1 s=29161 n(v0 rc2024-05-10T21:15:19.322491+0000 b29161 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48391080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023d1:[<0x200000023b5/article-58.xml v576>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023d1:[<0x200000023b5/article-58.xml v917>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000241c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml auth v857 ap=1 s=15577 n(v0 rc2024-05-10T21:15:20.110471+0000 b15577 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49312100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000241c:[<0x200000023b5/article-28.xml v714>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000241c:[<0x200000023b5/article-28.xml v857>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.686+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023e5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml auth v905 ap=1 s=27924 n(v0 rc2024-05-10T21:15:19.404489+0000 b27924 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d6a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023e5:[<0x200000023b5/article-52.xml v630>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023e5:[<0x200000023b5/article-52.xml v905>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024ae [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg auth v769 ap=1 s=45723 n(v0 rc2024-05-10T21:15:22.070423+0000 b45723 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d68580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024ae:[<0x200000023b5/picture-3.jpg v734>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024ae:[<0x200000023b5/picture-3.jpg v769>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002475 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg auth v801 ap=1 s=21644 n(v0 rc2024-05-10T21:15:20.958450+0000 b21644 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d69b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002475:[<0x200000023b5/picture-11.jpg v676>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002475:[<0x200000023b5/picture-11.jpg v801>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000244a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml auth v779 ap=1 s=12935 n(v0 rc2024-05-10T21:15:20.439463+0000 b12935 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d6b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000244a:[<0x200000023b5/article-5.xml v650>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000244a:[<0x200000023b5/article-5.xml v779>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.705 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.689+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002410 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml auth v869 ap=1 s=14033 n(v0 rc2024-05-10T21:15:20.012474+0000 b14033 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498eb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002410:[<0x200000023b5/article-34.xml v674>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002410:[<0x200000023b5/article-34.xml v869>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000241a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml auth v859 ap=1 s=21642 n(v0 rc2024-05-10T21:15:20.094472+0000 b21642 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498e8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000241a:[<0x200000023b5/article-29.xml v692>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000241a:[<0x200000023b5/article-29.xml v859>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.691+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023cb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml auth v921 ap=1 s=7922 n(v0 rc2024-05-10T21:15:19.297491+0000 b7922 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498e9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023cb:[<0x200000023b5/article-60.xml v628>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023cb:[<0x200000023b5/article-60.xml v921>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.692+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024fe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml auth v843 ap=1 s=23438 n(v0 rc2024-05-10T21:15:23.657383+0000 b23438 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498eb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024fe:[<0x200000023b5/article-22.xml v658>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024fe:[<0x200000023b5/article-22.xml v843>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023ed [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml auth v899 ap=1 s=4577 n(v0 rc2024-05-10T21:15:19.471487+0000 b4577 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498e8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023ed:[<0x200000023b5/article-49.xml v686>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023ed:[<0x200000023b5/article-49.xml v899>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002438 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml auth v815 ap=1 s=11950 n(v0 rc2024-05-10T21:15:20.289467+0000 b11950 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498eac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002438:[<0x200000023b5/article-14.xml v594>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002438:[<0x200000023b5/article-14.xml v815>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.694+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000257c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml auth v893 ap=1 s=12970 n(v0 rc2024-05-10T21:15:28.057274+0000 b12970 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff498e9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000257c:[<0x200000023b5/article-46.xml v748>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000257c:[<0x200000023b5/article-46.xml v893>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.697+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023f3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml auth v895 ap=1 s=30323 n(v0 rc2024-05-10T21:15:19.654483+0000 b30323 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48853080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023f3:[<0x200000023b5/article-47.xml v120>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023f3:[<0x200000023b5/article-47.xml v895>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.697+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000243e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml auth v803 ap=1 s=31772 n(v0 rc2024-05-10T21:15:20.333466+0000 b31772 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48854c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000243e:[<0x200000023b5/article-11.xml v644>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000243e:[<0x200000023b5/article-11.xml v803>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.698+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002434 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml auth v823 ap=1 s=6895 n(v0 rc2024-05-10T21:15:20.261467+0000 b6895 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48853b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002434:[<0x200000023b5/article-16.xml v602>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002434:[<0x200000023b5/article-16.xml v823>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023df [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml auth v909 ap=1 s=10830 n(v0 rc2024-05-10T21:15:19.366490+0000 b10830 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49e5ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023df:[<0x200000023b5/article-54.xml v606>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023df:[<0x200000023b5/article-54.xml v909>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.700+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002420 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml auth v851 ap=1 s=18195 n(v0 rc2024-05-10T21:15:20.134471+0000 b18195 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48bc1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002420:[<0x200000023b5/article-26.xml v592>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002420:[<0x200000023b5/article-26.xml v851>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.706 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.702+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002464 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg auth v817 ap=1 s=40325 n(v0 rc2024-05-10T21:15:20.715456+0000 b40325 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48bc2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002464:[<0x200000023b5/picture-15.jpg v668>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002464:[<0x200000023b5/picture-15.jpg v817>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.946 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002440 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml auth v799 ap=1 s=3274 n(v0 rc2024-05-10T21:15:20.345465+0000 b3274 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48bc0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002440:[<0x200000023b5/article-10.xml v654>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002440:[<0x200000023b5/article-10.xml v799>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002470 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg auth v805 ap=1 s=56012 n(v0 rc2024-05-10T21:15:20.849453+0000 b56012 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4856d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002470:[<0x200000023b5/picture-12.jpg v694>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002470:[<0x200000023b5/picture-12.jpg v805>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.705+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023f0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml auth v897 ap=1 s=30734 n(v0 rc2024-05-10T21:15:19.493487+0000 b30734 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4856f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023f0:[<0x200000023b5/article-48.xml v572>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023f0:[<0x200000023b5/article-48.xml v897>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.708+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000244c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml auth v775 ap=1 s=20481 n(v0 rc2024-05-10T21:15:20.456463+0000 b20481 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000244c:[<0x200000023b5/article-4.xml v622>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000244c:[<0x200000023b5/article-4.xml v775>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.708+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002422 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml auth v849 ap=1 s=22543 n(v0 rc2024-05-10T21:15:20.147470+0000 b22543 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a64580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002422:[<0x200000023b5/article-25.xml v610>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002422:[<0x200000023b5/article-25.xml v849>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.709+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000240e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml auth v871 ap=1 s=27407 n(v0 rc2024-05-10T21:15:19.979474+0000 b27407 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a65b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000240e:[<0x200000023b5/article-35.xml v582>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000240e:[<0x200000023b5/article-35.xml v871>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.711+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002404 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml auth v881 ap=1 s=18751 n(v0 rc2024-05-10T21:15:19.853478+0000 b18751 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002404:[<0x200000023b5/article-40.xml v696>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002404:[<0x200000023b5/article-40.xml v881>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.711+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002408 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml auth v877 ap=1 s=3983 n(v0 rc2024-05-10T21:15:19.881477+0000 b3983 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a66c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002408:[<0x200000023b5/article-38.xml v684>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002408:[<0x200000023b5/article-38.xml v877>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.713+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002436 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml auth v819 ap=1 s=32407 n(v0 rc2024-05-10T21:15:20.276467+0000 b32407 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002436:[<0x200000023b5/article-15.xml v618>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002436:[<0x200000023b5/article-15.xml v819>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.713+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000242e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml auth v835 ap=1 s=6952 n(v0 rc2024-05-10T21:15:20.228468+0000 b6952 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000242e:[<0x200000023b5/article-19.xml v638>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000242e:[<0x200000023b5/article-19.xml v835>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.715+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002455 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg auth v837 ap=1 s=39456 n(v0 rc2024-05-10T21:15:20.540461+0000 b39456 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002455:[<0x200000023b5/picture-20.jpg v634>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002455:[<0x200000023b5/picture-20.jpg v837>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.715+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002400 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml auth v885 ap=1 s=21430 n(v0 rc2024-05-10T21:15:19.827478+0000 b21430 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002400:[<0x200000023b5/article-42.xml v612>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002400:[<0x200000023b5/article-42.xml v885>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.947 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023bd [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml auth v929 ap=1 s=30751 n(v0 rc2024-05-10T21:15:19.254492+0000 b30751 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023bd:[<0x200000023b5/article-64.xml v588>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023bd:[<0x200000023b5/article-64.xml v929>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.717+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000240a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml auth v875 ap=1 s=9022 n(v0 rc2024-05-10T21:15:19.900476+0000 b9022 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff479b7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000240a:[<0x200000023b5/article-37.xml v700>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000240a:[<0x200000023b5/article-37.xml v875>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.720+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024f9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg auth v853 ap=1 s=24803 n(v0 rc2024-05-10T21:15:23.507387+0000 b24803 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47996580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024f9:[<0x200000023b5/picture-27.jpg v555>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024f9:[<0x200000023b5/picture-27.jpg v853>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.720+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002406 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml auth v879 ap=1 s=5877 n(v0 rc2024-05-10T21:15:19.860477+0000 b5877 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47996000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002406:[<0x200000023b5/article-39.xml v662>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002406:[<0x200000023b5/article-39.xml v879>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.722+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024ab [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml auth v873 ap=1 s=18677 n(v0 rc2024-05-10T21:15:22.049423+0000 b18677 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482fc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024ab:[<0x200000023b5/article-36.xml v724>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024ab:[<0x200000023b5/article-36.xml v873>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.722+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000243c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml auth v807 ap=1 s=7907 n(v0 rc2024-05-10T21:15:20.323466+0000 b7907 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482ff180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000243c:[<0x200000023b5/article-12.xml v716>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000243c:[<0x200000023b5/article-12.xml v807>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.948 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.725+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002453 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml auth v763 ap=1 s=7325 n(v0 rc2024-05-10T21:15:20.512461+0000 b7325 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497c6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002453:[<0x200000023b5/article-1.xml v728>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002453:[<0x200000023b5/article-1.xml v763>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.725+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000024c0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg auth v935 ap=1 s=20329 n(v0 rc2024-05-10T21:15:22.360416+0000 b20329 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497c6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000024c0:[<0x200000023b5/picture-67.jpg v730>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000024c0:[<0x200000023b5/picture-67.jpg v935>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.725+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023b6 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml auth v933 ap=1 s=12361 n(v0 rc2024-05-10T21:15:19.210494+0000 b12361 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497c8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023b6:[<0x200000023b5/article-66.xml v640>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023b6:[<0x200000023b5/article-66.xml v933>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.725+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002461 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg auth v821 ap=1 s=47783 n(v0 rc2024-05-10T21:15:20.676457+0000 b47783 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002461:[<0x200000023b5/picture-16.jpg v732>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002461:[<0x200000023b5/picture-16.jpg v821>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.727+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002426 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml auth v845 ap=1 s=1866 n(v0 rc2024-05-10T21:15:20.175470+0000 b1866 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002426:[<0x200000023b5/article-23.xml v698>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002426:[<0x200000023b5/article-23.xml v845>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.728+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002432 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml auth v827 ap=1 s=24667 n(v0 rc2024-05-10T21:15:20.251468+0000 b24667 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497f3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002432:[<0x200000023b5/article-17.xml v608>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002432:[<0x200000023b5/article-17.xml v827>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.730+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023c1 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml auth v927 ap=1 s=9850 n(v0 rc2024-05-10T21:15:19.264492+0000 b9850 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff497f0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023c1:[<0x200000023b5/article-63.xml v590>,<0x10000000399/blog-71 v93343>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023c1:[<0x200000023b5/article-63.xml v927>,<0x10000000399/blog-71 v129598>,<0x10000000208/blogtest_in v2291>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.941+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d84 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml auth v837 ap=1 s=13172 n(v0 rc2024-05-10T21:15:35.380093+0000 b13172 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a13b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d84:[<0x20000000834/article-24.xml v825>,<0x10000000399/blog-23 v94285>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d84:[<0x20000000834/article-24.xml v837>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.949 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.947+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d43 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml auth v837 ap=1 s=5276 n(v0 rc2024-05-10T21:15:29.439240+0000 b5276 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a1fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d43:[<0x20000000834/article-2.xml v812>,<0x10000000399/blog-23 v88033>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d43:[<0x20000000834/article-2.xml v837>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.949+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d8d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg auth v840 ap=1 s=32434 n(v0 rc2024-05-10T21:15:36.856057+0000 b32434 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49855600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d8d:[<0x20000000834/picture-69.jpg v820>,<0x10000000399/blog-23 v94285>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d8d:[<0x20000000834/picture-69.jpg v840>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.950 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.949+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000121e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg auth v835 ap=1 s=19480 n(v0 rc2024-05-10T21:16:13.033166+0000 b19480 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff482d4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000121e:[<0x20000000834/picture-46.jpg v829>,<0x10000000399/blog-23 v121311>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x1000000121e:[<0x20000000834/picture-46.jpg v835>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:34.950 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:17:35.261 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.988+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000daf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml auth v868 ap=1 s=14866 n(v0 rc2024-05-10T21:15:55.029608+0000 b14866 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482a1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000daf:[<0x20000000834/article-14.xml v833>,<0x10000000399/blog-23 v106653>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000000daf:[<0x20000000834/article-14.xml v868>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.261 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.991+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d8e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml auth v866 ap=1 s=30937 n(v0 rc2024-05-10T21:15:36.866056+0000 b30937 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4c640100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d8e:[<0x20000000834/article-54.xml v829>,<0x10000000399/blog-23 v94285>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d8e:[<0x20000000834/article-54.xml v866>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.261 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000287a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002901 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002897 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a46 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a4c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.262 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002973 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000296a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a34 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a4f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002924 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a40 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.263 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000293a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000297f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.991+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001723 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml auth v864 ap=1 s=13025 n(v0 rc2024-05-10T21:16:19.521006+0000 b13025 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4af50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001723:[<0x20000000834/article-93.xml v853>,<0x10000000399/blog-23 v127755>,<0x10000000208/blogtest_in v2107>,<0x10000000201/src v1784>,<0x10000000006/blogbench-1.0 v1565>,<0x10000000005/tmp v132>,<0x100000001fb/client.0 v133>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v145>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v162>,<0x1/volumes v151>]//[]","memoryvalue":"(3)0x10000001723:[<0x20000000834/article-93.xml v864>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.264 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.992+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001722 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg auth v862 ap=1 s=32772 n(v0 rc2024-05-10T21:16:19.514006+0000 b32772 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4adf4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001722:[<0x20000000834/picture-12.jpg v849>,<0x10000000399/blog-23 v127755>,<0x10000000208/blogtest_in v2107>,<0x10000000201/src v1784>,<0x10000000006/blogbench-1.0 v1565>,<0x10000000005/tmp v132>,<0x100000001fb/client.0 v133>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v145>,<0x10000000001/sv_1 v169>,<0x10000000000/qa v162>,<0x1/volumes v151>]//[]","memoryvalue":"(3)0x10000001722:[<0x20000000834/picture-12.jpg v862>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000292c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000295e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029ef (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029dc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002889 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002928 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029f5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000dab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002914 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002930 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.992+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001222 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml auth v856 ap=1 s=19639 n(v0 rc2024-05-10T21:16:13.047165+0000 b19639 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff48260b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001222:[<0x20000000834/article-38.xml v842>,<0x10000000399/blog-23 v121311>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x10000001222:[<0x20000000834/article-38.xml v856>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:34.994+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000db3 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml auth v838 ap=1 s=1890 n(v0 rc2024-05-10T21:15:55.235603+0000 b1890 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49d2a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000db3:[<0x20000000834/article-82.xml v823>,<0x10000000399/blog-23 v106653>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000000db3:[<0x20000000834/article-82.xml v838>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.266 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:35.000+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d89 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml auth v850 ap=1 s=20803 n(v0 rc2024-05-10T21:15:35.767084+0000 b20803 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4959c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d89:[<0x20000000834/article-17.xml v834>,<0x10000000399/blog-23 v94285>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d89:[<0x20000000834/article-17.xml v850>,<0x10000000399/blog-23 v130066>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000293d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000289b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029fb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000298d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002885 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028b1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029a5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028ce (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000297c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002881 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002967 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002890 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000287e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002894 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002943 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000295b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000299f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002970 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a49 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002985 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002958 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002934 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028cc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002955 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002876 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000296d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029d7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000289f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.270 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002910 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002976 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002964 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000291c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000288c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002908 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.271 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002982 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x1000000168d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028a1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002937 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002952 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000294f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002979 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002949 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.272 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029f8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000290c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029e0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.273 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002918 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002940 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000299a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002a22 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002961 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000294c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002993 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029f2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002946 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000028f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002905 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000029af (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.274 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022a6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000219f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000238e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023b2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002275 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000220f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023a3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000224a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000234d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.275 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002195 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000218e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021ca (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002332 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002368 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002282 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002358 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000233e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000227a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002290 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002323 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000232c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.276 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002191 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021d0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002223 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000221e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002350 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002226 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000226b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002304 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000233b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002287 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000236b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002294 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000224d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000225e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002341 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002311 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002f0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000039c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c7e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000003c67 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000003f83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: pgmap v723: 97 pgs: 97 active+clean; 443 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 896 KiB/s wr, 253 op/s 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cd2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000003b6c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c94 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.279 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000caf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c6e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cc2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cbd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c92 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c95 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000caa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.280 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c81 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c77 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c87 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c71 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c82 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c70 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cbf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cba (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c80 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c98 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001123 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cbe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cbc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001121 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c76 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001120 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c97 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001124 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c75 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c93 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c88 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001154 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001122 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c91 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c7d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cad (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c7c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.284 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c72 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c7b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c74 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cac (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c96 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c6f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cc0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.285 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cb3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cbb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cc1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c9b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000ca2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c8b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000c99 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000287a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002901 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002897 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a46 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a4c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002973 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000296a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a34 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a4f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002924 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a40 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000293a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000297f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000292c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000295e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029ef (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029dc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002889 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002928 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029f5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000dab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002914 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002930 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000293d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000289b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029fb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000298d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002885 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028b1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029a5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028ce (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000297c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002881 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002967 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002890 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000287e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002894 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002943 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000295b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000299f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002970 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a49 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002985 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002958 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002934 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028cc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002955 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002876 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000296d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029d7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000289f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002910 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002976 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002964 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000291c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000288c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002908 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002982 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.374 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x1000000168d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028a1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002937 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002952 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000294f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002979 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002949 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029f8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.375 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000290c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029e0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002918 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002940 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000299a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.376 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002a22 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002961 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000294c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002993 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029f2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002946 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000028f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002905 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000029af (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022a6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000219f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.377 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000238e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023b2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002275 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000220f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023a3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000224a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000234d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002195 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000218e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021ca (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002332 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002368 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.378 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002282 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002358 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000233e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000227a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002290 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002323 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000232c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002191 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021d0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002223 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000221e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002350 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.379 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002226 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000226b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002304 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000233b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002287 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000236b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002294 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000224d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000225e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002341 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002311 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.380 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002f0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000039c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c7e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000003c67 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000003f83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: pgmap v723: 97 pgs: 97 active+clean; 443 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 896 KiB/s wr, 253 op/s 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cd2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000003b6c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c94 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000caf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c6e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cc2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cbd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c92 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c95 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000caa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c81 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c77 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c87 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c71 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c82 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c70 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cbf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.383 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cba (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c80 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c98 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001123 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cbe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cbc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001121 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.384 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c76 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001120 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c97 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001124 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c75 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c93 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c88 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001154 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001122 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.385 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c91 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c7d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cad (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c7c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c72 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c7b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.386 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c74 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cac (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c96 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c6f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cc0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cb3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.387 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cbb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cc1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c9b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000ca2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c8b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:35 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000c99 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000000bd9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000287a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002901 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002897 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a46 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a4c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.391 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002973 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000296a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a34 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a4f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002924 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a40 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000293a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000297f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000292c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000295e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029ef (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029dc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002889 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002928 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029aa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029f5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.393 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000dab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002914 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002930 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000293d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000289b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029fb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000298d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002885 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028b1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.394 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029a5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028ce (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000297c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002881 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002967 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002890 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000287e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002894 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002943 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000295b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000299f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002970 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a49 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.395 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002985 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002958 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002934 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028cc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002955 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002876 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000296d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029d7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000289f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002910 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.396 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002976 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002964 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000291c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000288c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002908 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002982 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x1000000168d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.397 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028a1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002937 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028e9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002952 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000294f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002979 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002949 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029f8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000290c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.398 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029e6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029c8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029e0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028be (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002918 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002940 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000299a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029ec (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002a22 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.399 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002961 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000294c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002993 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029f2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002946 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000028f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002905 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000029af (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022a6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000219f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000238e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023b2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.400 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002275 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000220f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023a3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000224a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000234d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002195 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000218e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021ca (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002332 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002368 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002282 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002358 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000233e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.401 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022d8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000227a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002290 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002323 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000232c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002191 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021d0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002223 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021b4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000221e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002350 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002226 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000226b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.402 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002304 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000233b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002287 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000236b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002294 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021a8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000224d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021cd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000225e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002341 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021d2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002311 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002f0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000039c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.403 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c7e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000003c67 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000003f83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: pgmap v723: 97 pgs: 97 active+clean; 443 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 896 KiB/s wr, 253 op/s 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cd2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000003b6c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c94 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001125 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000caf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c6e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.404 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cc2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c85 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cbd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c92 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c95 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000caa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c81 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.405 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c86 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c77 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c87 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c71 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c82 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c70 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cbf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cba (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.406 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c80 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c98 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001123 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cbe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cbc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001121 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c76 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001120 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.407 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c97 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001124 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c75 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c93 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c88 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001154 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001122 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c91 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c7d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.408 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cad (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c83 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c90 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c7c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c72 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c7b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.409 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c74 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cac (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c96 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c6f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cc0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cb3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d79 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.410 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cbb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cc1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d78 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c9b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c73 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000ca2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c8b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.411 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:35 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000c99 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:35.518 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:35.408+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d3d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml auth v1597 ap=1 s=28216 n(v0 rc2024-05-10T21:15:24.972351+0000 b28216 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff47f1a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d3d:[<0x2000000050a/article-80.xml v1593>,<0x10000000399/blog-17 v92874>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d3d:[<0x2000000050a/article-80.xml v1597>,<0x10000000399/blog-17 v129843>,<0x10000000208/blogtest_in v2293>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:35.521+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000003bdf [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml auth v1616 ap=1 s=23001 n(v0 rc2024-05-10T21:16:10.881218+0000 b23001 1=1+0) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=0 freezing=0 frozen=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55ff4bef3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000003bdf:[<0x2000000050a/article-96.xml v1606>,<0x10000000399/blog-17 v107039>,<0x10000000208/blogtest_in v2006>,<0x10000000201/src v1780>,<0x10000000006/blogbench-1.0 v1561>,<0x10000000005/tmp v128>,<0x100000001fb/client.0 v129>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v141>,<0x10000000001/sv_1 v165>,<0x10000000000/qa v158>,<0x1/volumes v147>]//[]","memoryvalue":"(3)0x20000003bdf:[<0x2000000050a/article-96.xml v1616>,<0x10000000399/blog-17 v129843>,<0x10000000208/blogtest_in v2295>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:35.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:35.591+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x300000004ea [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml auth{2=1} v1603 ap=1 s=9004 n(v0 rc2024-05-10T21:15:19.661482+0000 b9004 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55ff4a31c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x300000004ea:[<0x2000000050a/comment-3.xml v1581>,<0x10000000399/blog-17 v92874>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x300000004ea:[<0x2000000050a/comment-3.xml v1603>,<0x10000000399/blog-17 v129843>,<0x10000000208/blogtest_in v2295>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000246b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000245a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002430 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002457 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023e8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000241e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000242c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002467 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002402 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000248d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000249e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002416 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.306 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002442 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023fc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002495 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023c5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002424 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002577 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000245e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000244e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002446 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002414 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000242a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.308 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002488 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002444 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000243a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002418 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002448 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023eb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002451 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002412 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000249d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023d1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000241c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002475 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000244a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002410 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.310 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000241a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023cb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023ed (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002438 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000257c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000243e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002434 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002420 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002464 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002440 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002470 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000244c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002422 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000240e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002404 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002408 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002436 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000242e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002455 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002400 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023bd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000240a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024f9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002406 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000243c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002453 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000024c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023b6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002461 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002426 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002432 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x1000000121e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000daf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001723 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001722 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000001222 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000db3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000246b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000245a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002430 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002457 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023e8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000241e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000242c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002467 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002402 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000248d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000249e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002416 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002442 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023fc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002495 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023c5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002424 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002577 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000245e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000244e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002446 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002414 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000242a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002488 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002444 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000243a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002418 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002448 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023eb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002451 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002412 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000249d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023d1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000241c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002475 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000244a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002410 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000241a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023cb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023ed (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002438 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000257c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000243e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002434 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002420 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002464 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002440 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002470 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000244c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002422 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000240e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002404 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002408 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002436 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000242e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002455 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.370 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002400 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023bd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000240a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024f9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002406 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000243c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002453 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.371 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000024c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023b6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002461 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002426 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002432 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x1000000121e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000daf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001723 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001722 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000001222 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.372 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000db3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.373 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:36 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023cf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000246b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000245a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002430 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002457 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023e8 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000241e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024b9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000242c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002467 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002402 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023fa (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000248d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002483 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000249e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024c3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002416 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002442 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023fc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002495 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023c5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002424 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002577 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000245e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000244e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002446 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023db (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002414 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000242a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002488 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002444 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000243a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002418 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002448 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023eb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002451 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002412 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000249d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023d5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023e3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023d1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000241c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024ae (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002475 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000244a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002410 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000241a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023cb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024fe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023ed (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002438 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000257c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023f3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000243e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002434 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023df (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002420 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002464 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002440 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002470 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023f0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000244c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002422 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000240e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002404 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002408 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002436 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000242e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002455 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002400 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023bd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000240a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024f9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002406 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024ab (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000243c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002453 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000024c0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023b6 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002461 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002426 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002432 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023c1 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d84 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d43 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d8d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x1000000121e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000daf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d8e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001723 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001722 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000001222 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000db3 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d89 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.383 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:36 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d3d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.613+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cfc [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml auth v1431 ap=1 s=7807 n(v0 rc2024-05-10T21:15:22.061423+0000 b7807 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482d2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cfc:[<0x20000000adc/comment-56.xml v1379>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cfc:[<0x20000000adc/comment-56.xml v1431>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.616+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d38 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml auth v1433 ap=1 s=2452 n(v0 rc2024-05-10T21:15:24.424364+0000 b2452 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a77600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d38:[<0x20000000adc/comment-5.xml v1400>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d38:[<0x20000000adc/comment-5.xml v1433>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.619+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d18 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml auth v1429 ap=1 s=3760 n(v0 rc2024-05-10T21:15:22.685407+0000 b3760 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47884b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d18:[<0x20000000adc/comment-29.xml v1377>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d18:[<0x20000000adc/comment-29.xml v1429>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.623+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000da5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml auth v1437 ap=1 s=20840 n(v0 rc2024-05-10T21:15:49.805737+0000 b20840 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a07db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000da5:[<0x20000000adc/article-20.xml v1419>,<0x10000000399/blog-27 v107051>,<0x10000000208/blogtest_in v1825>,<0x10000000201/src v1776>,<0x10000000006/blogbench-1.0 v1557>,<0x10000000005/tmp v124>,<0x100000001fb/client.0 v125>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v137>,<0x10000000001/sv_1 v161>,<0x10000000000/qa v154>,<0x1/volumes v143>]//[]","memoryvalue":"(3)0x10000000da5:[<0x20000000adc/article-20.xml v1437>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.627+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml auth v1435 ap=1 s=4361 n(v0 rc2024-05-10T21:15:23.852379+0000 b4361 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ca32c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2e:[<0x20000000adc/comment-11.xml v1385>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2e:[<0x20000000adc/comment-11.xml v1435>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.629+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d23 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml auth v1427 ap=1 s=7552 n(v0 rc2024-05-10T21:15:22.907402+0000 b7552 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4ca33180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d23:[<0x20000000adc/comment-20.xml v1383>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d23:[<0x20000000adc/comment-20.xml v1427>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.915 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.632+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d05 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml auth v1434 ap=1 s=9232 n(v0 rc2024-05-10T21:15:22.145421+0000 b9232 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48294000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d05:[<0x20000000adc/comment-47.xml v1397>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d05:[<0x20000000adc/comment-47.xml v1434>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.632+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d06 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml auth v1432 ap=1 s=5732 n(v0 rc2024-05-10T21:15:22.164420+0000 b5732 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48280680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d06:[<0x20000000adc/comment-46.xml v1387>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d06:[<0x20000000adc/comment-46.xml v1432>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.640+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d10 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml auth v1430 ap=1 s=3937 n(v0 rc2024-05-10T21:15:22.464413+0000 b3937 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47ba7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d10:[<0x20000000adc/comment-36.xml v1393>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d10:[<0x20000000adc/comment-36.xml v1430>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.643+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d39 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml auth v1436 ap=1 s=3382 n(v0 rc2024-05-10T21:15:24.647359+0000 b3382 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47ba8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d39:[<0x20000000adc/comment-4.xml v1414>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d39:[<0x20000000adc/comment-4.xml v1436>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.647+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml auth v1438 ap=1 s=5295 n(v0 rc2024-05-10T21:15:23.721382+0000 b5295 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47ba6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2b:[<0x20000000adc/comment-14.xml v1395>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2b:[<0x20000000adc/comment-14.xml v1438>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.650+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d24 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml auth v1428 ap=1 s=9094 n(v0 rc2024-05-10T21:15:23.028399+0000 b9094 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cc7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d24:[<0x20000000adc/comment-19.xml v1389>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d24:[<0x20000000adc/comment-19.xml v1428>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.916 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.650+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml auth v1433 ap=1 s=3495 n(v0 rc2024-05-10T21:15:22.825404+0000 b3495 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cc4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1f:[<0x20000000adc/comment-22.xml v1393>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1f:[<0x20000000adc/comment-22.xml v1433>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.917 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.651+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d3b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml auth v1445 ap=1 s=6255 n(v0 rc2024-05-10T21:15:24.743357+0000 b6255 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49cc7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d3b:[<0x20000000adc/comment-2.xml v1417>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d3b:[<0x20000000adc/comment-2.xml v1445>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.917 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.651+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d08 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml auth v1439 ap=1 s=9925 n(v0 rc2024-05-10T21:15:22.196420+0000 b9925 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48f97b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d08:[<0x20000000adc/comment-44.xml v1401>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d08:[<0x20000000adc/comment-44.xml v1439>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.917 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.657+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml auth v1435 ap=1 s=8336 n(v0 rc2024-05-10T21:15:22.710407+0000 b8336 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2e1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1a:[<0x20000000adc/comment-27.xml v1405>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1a:[<0x20000000adc/comment-27.xml v1435>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.917 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.658+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d00 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml auth v1443 ap=1 s=7104 n(v0 rc2024-05-10T21:15:22.097422+0000 b7104 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4a2e3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d00:[<0x20000000adc/comment-52.xml v1409>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d00:[<0x20000000adc/comment-52.xml v1443>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.661+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d32 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml auth v1447 ap=1 s=8801 n(v0 rc2024-05-10T21:15:24.015375+0000 b8801 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff480cb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d32:[<0x20000000adc/comment-7.xml v1395>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d32:[<0x20000000adc/comment-7.xml v1447>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.662+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml auth v1451 ap=1 s=5687 n(v0 rc2024-05-10T21:15:23.673383+0000 b5687 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff480cc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2a:[<0x20000000adc/comment-15.xml v1397>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2a:[<0x20000000adc/comment-15.xml v1451>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.664+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d09 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml auth v1437 ap=1 s=4186 n(v0 rc2024-05-10T21:15:22.208419+0000 b4186 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47eaa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d09:[<0x20000000adc/comment-43.xml v1399>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d09:[<0x20000000adc/comment-43.xml v1437>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.666+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d28 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml auth v1431 ap=1 s=3145 n(v0 rc2024-05-10T21:15:23.582385+0000 b3145 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47eaac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d28:[<0x20000000adc/comment-17.xml v1371>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d28:[<0x20000000adc/comment-17.xml v1431>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.668+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d30 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml auth v1449 ap=1 s=6754 n(v0 rc2024-05-10T21:15:23.938376+0000 b6754 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482bc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d30:[<0x20000000adc/comment-9.xml v1403>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d30:[<0x20000000adc/comment-9.xml v1449>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.669+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d07 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml auth v1441 ap=1 s=2299 n(v0 rc2024-05-10T21:15:22.182420+0000 b2299 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482bc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d07:[<0x20000000adc/comment-45.xml v1411>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d07:[<0x20000000adc/comment-45.xml v1441>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.671+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml auth v1427 ap=1 s=9062 n(v0 rc2024-05-10T21:15:23.893378+0000 b9062 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47894100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2f:[<0x20000000adc/comment-10.xml v1397>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2f:[<0x20000000adc/comment-10.xml v1427>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.673+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cff [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml auth v1421 ap=1 s=4477 n(v0 rc2024-05-10T21:15:22.088422+0000 b4477 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49ed1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cff:[<0x20000000adc/comment-53.xml v1389>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cff:[<0x20000000adc/comment-53.xml v1421>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.673+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d37 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml auth v1423 ap=1 s=9935 n(v0 rc2024-05-10T21:15:24.023374+0000 b9935 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4961f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d37:[<0x20000000adc/comment-6.xml v1386>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d37:[<0x20000000adc/comment-6.xml v1423>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.918 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d11 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml auth v1417 ap=1 s=6879 n(v0 rc2024-05-10T21:15:22.506412+0000 b6879 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4961f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d11:[<0x20000000adc/comment-35.xml v1391>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d11:[<0x20000000adc/comment-35.xml v1417>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.919 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.679+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d19 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml auth v1415 ap=1 s=2555 n(v0 rc2024-05-10T21:15:22.693407+0000 b2555 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47cc7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d19:[<0x20000000adc/comment-28.xml v1395>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d19:[<0x20000000adc/comment-28.xml v1415>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.919 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.685+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml auth v1419 ap=1 s=5337 n(v0 rc2024-05-10T21:15:22.413414+0000 b5337 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff474aac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0e:[<0x20000000adc/comment-38.xml v1393>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0e:[<0x20000000adc/comment-38.xml v1419>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.919 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.687+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d31 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml auth v1425 ap=1 s=6596 n(v0 rc2024-05-10T21:15:23.945376+0000 b6596 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff474aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d31:[<0x20000000adc/comment-8.xml v1399>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d31:[<0x20000000adc/comment-8.xml v1425>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.919 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.701+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d02 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml auth v1377 ap=1 s=9202 n(v0 rc2024-05-10T21:15:22.114422+0000 b9202 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4799f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d02:[<0x20000000adc/comment-50.xml v1359>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d02:[<0x20000000adc/comment-50.xml v1377>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.919 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.702+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d04 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml auth v1375 ap=1 s=6508 n(v0 rc2024-05-10T21:15:22.136421+0000 b6508 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4799eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d04:[<0x20000000adc/comment-48.xml v1363>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d04:[<0x20000000adc/comment-48.xml v1375>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.920 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.706+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d15 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml auth v1373 ap=1 s=4687 n(v0 rc2024-05-10T21:15:22.648408+0000 b4687 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48c6d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d15:[<0x20000000adc/comment-31.xml v1361>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d15:[<0x20000000adc/comment-31.xml v1373>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.920 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.708+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d3c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml auth v1451 ap=1 s=8794 n(v0 rc2024-05-10T21:15:24.769356+0000 b8794 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff485e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d3c:[<0x20000000adc/comment-1.xml v1427>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d3c:[<0x20000000adc/comment-1.xml v1451>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.920 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.710+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml auth v1447 ap=1 s=7031 n(v0 rc2024-05-10T21:15:22.357416+0000 b7031 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49003080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0c:[<0x20000000adc/comment-40.xml v1361>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0c:[<0x20000000adc/comment-40.xml v1447>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.920 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.710+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d13 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml auth v1443 ap=1 s=10211 n(v0 rc2024-05-10T21:15:22.587410+0000 b10211 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49004c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d13:[<0x20000000adc/comment-33.xml v1357>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d13:[<0x20000000adc/comment-33.xml v1443>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.920 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.711+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml auth v1445 ap=1 s=2756 n(v0 rc2024-05-10T21:15:22.399415+0000 b2756 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49003b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0d:[<0x20000000adc/comment-39.xml v1351>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0d:[<0x20000000adc/comment-39.xml v1445>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.712+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d14 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml auth v1441 ap=1 s=3026 n(v0 rc2024-05-10T21:15:22.617409+0000 b3026 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49002b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d14:[<0x20000000adc/comment-32.xml v1355>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d14:[<0x20000000adc/comment-32.xml v1441>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.712+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cfe [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml auth v1449 ap=1 s=5059 n(v0 rc2024-05-10T21:15:22.077423+0000 b5059 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48a77b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cfe:[<0x20000000adc/comment-54.xml v1353>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cfe:[<0x20000000adc/comment-54.xml v1449>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.826+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml auth v1459 ap=1 s=6690 n(v0 rc2024-05-10T21:15:23.729382+0000 b6690 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff496a4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2c:[<0x20000000adc/comment-13.xml v1407>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2c:[<0x20000000adc/comment-13.xml v1459>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.826+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d20 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml auth v1443 ap=1 s=8549 n(v0 rc2024-05-10T21:15:22.853403+0000 b8549 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff496a3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d20:[<0x20000000adc/comment-21.xml v1421>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d20:[<0x20000000adc/comment-21.xml v1443>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.826+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml auth v1445 ap=1 s=8923 n(v0 rc2024-05-10T21:15:22.720407+0000 b8923 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff496a2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1b:[<0x20000000adc/comment-26.xml v1405>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1b:[<0x20000000adc/comment-26.xml v1445>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.827+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d3a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml auth v1455 ap=1 s=10074 n(v0 rc2024-05-10T21:15:24.732357+0000 b10074 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d02580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d3a:[<0x20000000adc/comment-3.xml v1423>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d3a:[<0x20000000adc/comment-3.xml v1455>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.829+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d25 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml auth v1441 ap=1 s=3395 n(v0 rc2024-05-10T21:15:23.209395+0000 b3395 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48d02000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d25:[<0x20000000adc/comment-18.xml v1411>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d25:[<0x20000000adc/comment-18.xml v1441>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.829+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d03 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml auth v1451 ap=1 s=7682 n(v0 rc2024-05-10T21:15:22.127421+0000 b7682 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47cc8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d03:[<0x20000000adc/comment-49.xml v1419>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d03:[<0x20000000adc/comment-49.xml v1451>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.830+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d2d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml auth v1457 ap=1 s=3123 n(v0 rc2024-05-10T21:15:23.805380+0000 b3123 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47cc9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d2d:[<0x20000000adc/comment-12.xml v1413>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d2d:[<0x20000000adc/comment-12.xml v1457>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.830+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d16 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml auth v1447 ap=1 s=8889 n(v0 rc2024-05-10T21:15:22.678408+0000 b8889 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff494aa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d16:[<0x20000000adc/comment-30.xml v1409>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d16:[<0x20000000adc/comment-30.xml v1447>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.831+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml auth v1449 ap=1 s=6637 n(v0 rc2024-05-10T21:15:22.279418+0000 b6637 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47cc8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0a:[<0x20000000adc/comment-42.xml v1417>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0a:[<0x20000000adc/comment-42.xml v1449>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2299>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:36.921 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:36.850+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d01 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml auth v1453 ap=1 s=2257 n(v0 rc2024-05-10T21:15:22.106422+0000 b2257 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47cc6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d01:[<0x20000000adc/comment-51.xml v1415>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d01:[<0x20000000adc/comment-51.xml v1453>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2301>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000003bdf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-mon[29546]: Scrub error on inode 0x300000004ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-mon[29546]: pgmap v724: 97 pgs: 97 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 806 KiB/s wr, 378 op/s 2024-05-10T21:17:37.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:37 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000003bdf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:37 smithi092 ceph-mon[34415]: Scrub error on inode 0x300000004ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:37 smithi092 ceph-mon[34415]: pgmap v724: 97 pgs: 97 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 806 KiB/s wr, 378 op/s 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 -- 172.21.15.50:0/1769690583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105500 msgr2=0x7fcc14105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/1769690583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105500 0x7fcc14105910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009990 tx=0x7fcc04009ca0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 -- 172.21.15.50:0/1769690583 shutdown_connections 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/1769690583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc1410e8e0 0x7fcc14110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/1769690583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105500 0x7fcc14105910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/1769690583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc14105e50 0x7fcc1410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 -- 172.21.15.50:0/1769690583 >> 172.21.15.50:0/1769690583 conn(0x7fcc140fedc0 msgr2=0x7fcc141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 -- 172.21.15.50:0/1769690583 shutdown_connections 2024-05-10T21:17:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.373+0000 7fcc1aa00700 1 -- 172.21.15.50:0/1769690583 wait complete. 2024-05-10T21:17:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 Processor -- start 2024-05-10T21:17:37.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 -- start start 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 0x7fcc1419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 0x7fcc1419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 0x7fcc1419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14114480 con 0x7fcc14105e50 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14114300 con 0x7fcc1410e8e0 2024-05-10T21:17:37.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.375+0000 7fcc1aa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14114600 con 0x7fcc14105500 2024-05-10T21:17:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 0x7fcc1419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 0x7fcc1419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46118/0 (socket says 172.21.15.50:46118) 2024-05-10T21:17:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13400700 1 -- 172.21.15.50:0/3546716965 learned_addr learned my addr 172.21.15.50:0/3546716965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 0x7fcc1419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 0x7fcc1419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 -- 172.21.15.50:0/3546716965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 msgr2=0x7fcc1419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 0x7fcc1419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 -- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 msgr2=0x7fcc1419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 0x7fcc1419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc13e00700 1 -- 172.21.15.50:0/3546716965 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1419fcc0 con 0x7fcc14105500 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.376+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 0x7fcc1419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.377+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 0x7fcc1419adf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098e0 tx=0x7fcc040057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.377+0000 7fcc10c00700 1 -- 172.21.15.50:0/3546716965 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04027070 con 0x7fcc14105500 2024-05-10T21:17:37.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.377+0000 7fcc10c00700 1 -- 172.21.15.50:0/3546716965 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc04022470 con 0x7fcc14105500 2024-05-10T21:17:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.377+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1419ff50 con 0x7fcc14105500 2024-05-10T21:17:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.377+0000 7fcc10c00700 1 -- 172.21.15.50:0/3546716965 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04017650 con 0x7fcc14105500 2024-05-10T21:17:37.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.378+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc141a78d0 con 0x7fcc14105500 2024-05-10T21:17:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:37 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000003bdf (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:37 smithi142 ceph-mon[34846]: Scrub error on inode 0x300000004ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:37 smithi142 ceph-mon[34846]: pgmap v724: 97 pgs: 97 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 806 KiB/s wr, 378 op/s 2024-05-10T21:17:37.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.381+0000 7fcc10c00700 1 -- 172.21.15.50:0/3546716965 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcc0402a070 con 0x7fcc14105500 2024-05-10T21:17:37.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.383+0000 7fcc10c00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 0x7fcbf80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.383+0000 7fcc10c00700 1 -- 172.21.15.50:0/3546716965 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fcc040e2ed0 con 0x7fcc14105500 2024-05-10T21:17:37.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.383+0000 7fcc13400700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 0x7fcbf80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.384+0000 7fcc13400700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 0x7fcbf80b4e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009680 tx=0x7fcc00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.385+0000 7fcc29517b80 1 Processor -- start 2024-05-10T21:17:37.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.385+0000 7fcc29517b80 1 -- start start 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886da30e40 0x55886dbb46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55886dad0a20 con 0x55886dbaec50 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55886da6e760 con 0x55886dbb4bf0 2024-05-10T21:17:37.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc29517b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55886dacc9e0 con 0x55886da30e40 2024-05-10T21:17:37.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc13400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc13400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49874/0 (socket says 172.21.15.50:49874) 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc13400700 1 -- 172.21.15.50:0/3511583898 learned_addr learned my addr 172.21.15.50:0/3511583898 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.386+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886da30e40 0x55886dbb46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 -- 172.21.15.50:0/3511583898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886da30e40 msgr2=0x55886dbb46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886da30e40 0x55886dbb46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 -- 172.21.15.50:0/3511583898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 msgr2=0x55886dbaf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbaf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 -- 172.21.15.50:0/3511583898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886da1d8d0 con 0x55886dbb4bf0 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbaf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:17:37.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.387+0000 7fcc13400700 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc0001c570 tx=0x7fcc00014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e2aa0dda8d1ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.388+0000 7fcc09600700 1 -- 172.21.15.50:0/3511583898 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00027430 con 0x55886dbb4bf0 2024-05-10T21:17:37.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.388+0000 7fcc09600700 1 -- 172.21.15.50:0/3511583898 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc00027a80 con 0x55886dbb4bf0 2024-05-10T21:17:37.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.388+0000 7fcc09600700 1 -- 172.21.15.50:0/3511583898 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0001dab0 con 0x55886dbb4bf0 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 -- 172.21.15.50:0/3511583898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 msgr2=0x55886dbae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc0001c570 tx=0x7fcc00014000 comp rx=0 tx=0).stop 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 -- 172.21.15.50:0/3511583898 shutdown_connections 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbaf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886da30e40 0x55886dbb46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3511583898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbb4bf0 0x55886dbae710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.389+0000 7fcc29517b80 1 -- 172.21.15.50:0/3511583898 >> 172.21.15.50:0/3511583898 conn(0x55886da6fa80 msgr2=0x55886dba9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.390+0000 7fcc29517b80 1 -- 172.21.15.50:0/3511583898 shutdown_connections 2024-05-10T21:17:37.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.390+0000 7fcc29517b80 1 -- 172.21.15.50:0/3511583898 wait complete. 2024-05-10T21:17:37.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.390+0000 7fcc29517b80 1 -- start start 2024-05-10T21:17:37.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 Processor -- start 2024-05-10T21:17:37.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 0x55886dbb88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbb4bf0 0x55886dbb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55886dbbb6c0 con 0x55886dbb4bf0 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55886da36b20 con 0x55886da30e40 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.391+0000 7fcc29517b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55886dbbb840 con 0x55886dbaec50 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc29517b80 10 client.0 _ll_register_callbacks cb 0x55886da64380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 0x55886dbb88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 0x55886dbb88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50592/0 (socket says 172.21.15.50:50592) 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13400700 1 -- 172.21.15.50:0/1279721661 learned_addr learned my addr 172.21.15.50:0/1279721661 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc18a00700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbb4bf0 0x55886dbb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 -- 172.21.15.50:0/1279721661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 msgr2=0x55886dbb88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 0x55886dbb88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 -- 172.21.15.50:0/1279721661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbb4bf0 msgr2=0x55886dbb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbb4bf0 0x55886dbb8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.392+0000 7fcc13e00700 1 -- 172.21.15.50:0/1279721661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886da96420 con 0x55886da30e40 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcc13e00700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbb8390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc040a77c0 tx=0x7fcc040aa060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f323307049f95d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcbfe000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:37.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc040e0e60 con 0x55886da30e40 2024-05-10T21:17:37.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fcc040094b0 con 0x55886da30e40 2024-05-10T21:17:37.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc00027a80 con 0x55886da30e40 2024-05-10T21:17:37.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.393+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcc040ae900 con 0x55886da30e40 2024-05-10T21:17:37.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 10 client.26843 fetch_fsmap learned FSMap version 39 2024-05-10T21:17:37.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55886dbca610 con 0x55886da30e40 2024-05-10T21:17:37.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc00027a80 con 0x55886da30e40 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcc040b2f20 con 0x55886da30e40 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 10 client.26843 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 10 client.26843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 20 client.26843 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dbccb30 0x55886dbcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 4 client.26843 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.394+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886db7b070 con 0x55886dbccb30 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.395+0000 7fcc13400700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dbccb30 0x55886dbcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.395+0000 7fcc13400700 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dbccb30 0x55886dbcefc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825f92779795ce51 server_cookie=764af84a0ebb64f9 in_seq=0 out_seq=0 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.395+0000 7fcbfe000700 10 client.26843 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:37.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.396+0000 7fcbfe000700 1 -- 172.21.15.50:0/1279721661 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55886db7b070 con 0x55886dbccb30 2024-05-10T21:17:37.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.396+0000 7fcbfe000700 10 client.26843 handle_command_reply: tid=0 2024-05-10T21:17:37.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.396+0000 7fcc29517b80 1 client.26843 shutdown 2024-05-10T21:17:37.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 msgr2=0x55886dbb8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbb8390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc040a77c0 tx=0x7fcc040aa060 comp rx=0 tx=0).stop 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 shutdown_connections 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dbccb30 0x55886dbcefc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbb8390 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbb4bf0 0x55886dbb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 --2- 172.21.15.50:0/1279721661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbaec50 0x55886dbb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 >> 172.21.15.50:0/1279721661 conn(0x55886da6fa80 msgr2=0x55886dbad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcbfe000700 0 client.26843 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 shutdown_connections 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.397+0000 7fcc29517b80 1 -- 172.21.15.50:0/1279721661 wait complete. 2024-05-10T21:17:37.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.398+0000 7fcc29517b80 20 client.26843 trim_cache size 0 max 16384 2024-05-10T21:17:37.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.422+0000 7fcc29517b80 1 Processor -- start 2024-05-10T21:17:37.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.422+0000 7fcc29517b80 1 -- start start 2024-05-10T21:17:37.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 0x55886dbb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dbb72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141af1b0 con 0x55886da30e40 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ac980 con 0x55886dbaec50 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc29517b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141aa830 con 0x55886dbb4bf0 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc13e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc13e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46146/0 (socket says 172.21.15.50:46146) 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc13400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 0x55886dbb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc13e00700 1 -- 172.21.15.50:0/861986155 learned_addr learned my addr 172.21.15.50:0/861986155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.423+0000 7fcc18a00700 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dbb72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 -- 172.21.15.50:0/861986155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 msgr2=0x55886dbb72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dbb72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 -- 172.21.15.50:0/861986155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 msgr2=0x55886dbae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 -- 172.21.15.50:0/861986155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886dbb50a0 con 0x55886dbaec50 2024-05-10T21:17:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13e00700 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:37.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.424+0000 7fcc13400700 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 0x55886dbb2b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55886dbcf720 tx=0x7fcc000092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd511729d85a5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.425+0000 7fcc08c00700 1 -- 172.21.15.50:0/861986155 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0003a070 con 0x55886dbaec50 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.425+0000 7fcc08c00700 1 -- 172.21.15.50:0/861986155 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc0001d410 con 0x55886dbaec50 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.425+0000 7fcc08c00700 1 -- 172.21.15.50:0/861986155 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00027690 con 0x55886dbaec50 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 -- 172.21.15.50:0/861986155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 msgr2=0x55886dbb2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 0x55886dbb2b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55886dbcf720 tx=0x7fcc000092d0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 -- 172.21.15.50:0/861986155 shutdown_connections 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dbb72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886da30e40 0x55886dbae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 --2- 172.21.15.50:0/861986155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886dbaec50 0x55886dbb2b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 -- 172.21.15.50:0/861986155 >> 172.21.15.50:0/861986155 conn(0x55886da6fa80 msgr2=0x55886dba8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 -- 172.21.15.50:0/861986155 shutdown_connections 2024-05-10T21:17:37.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.426+0000 7fcc29517b80 1 -- 172.21.15.50:0/861986155 wait complete. 2024-05-10T21:17:37.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.427+0000 7fcc29517b80 1 -- start start 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.427+0000 7fcc29517b80 1 Processor -- start 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbd4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dc36330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141aae80 con 0x55886dbaec50 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ac2b0 con 0x55886da30e40 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ab510 con 0x55886dbb4bf0 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.428+0000 7fcc29517b80 10 client.0 _ll_register_callbacks cb 0x55886da64380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49894/0 (socket says 172.21.15.50:49894) 2024-05-10T21:17:37.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 -- 172.21.15.50:0/3336252572 learned_addr learned my addr 172.21.15.50:0/3336252572 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:37.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13400700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbd4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dc36330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 -- 172.21.15.50:0/3336252572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 msgr2=0x55886dc36330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dc36330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 -- 172.21.15.50:0/3336252572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 msgr2=0x55886dbd4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbd4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 -- 172.21.15.50:0/3336252572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886dbd5070 con 0x55886da30e40 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc18a00700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dc36330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcc13e00700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcc040a6af0 tx=0x7fcc040aa610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18ebb71dd304846 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.429+0000 7fcbfe000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc040e0e60 con 0x55886da30e40 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fcc040aa7b0 con 0x55886da30e40 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0001d410 con 0x55886da30e40 2024-05-10T21:17:37.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcc040aa9b0 con 0x55886da30e40 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcc29517b80 10 client.26849 fetch_fsmap learned FSMap version 39 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.430+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55886dc37850 con 0x55886da30e40 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc0001d410 con 0x55886da30e40 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcc040a97b0 con 0x55886da30e40 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 10 client.26849 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 20 client.26849 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dc37e50 0x55886dc3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 4 client.26849 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.431+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55886dade400 con 0x55886dc37e50 2024-05-10T21:17:37.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcc13400700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dc37e50 0x55886dc3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcc13400700 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dc37e50 0x55886dc3a270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e038d328e47e7b56 server_cookie=2c7157fd58cad8aa in_seq=0 out_seq=0 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcbfe000700 10 client.26849 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcbfe000700 1 -- 172.21.15.50:0/3336252572 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55886dade400 con 0x55886dc37e50 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcbfe000700 10 client.26849 handle_command_reply: tid=0 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.432+0000 7fcc29517b80 1 client.26849 shutdown 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 msgr2=0x55886dbd42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcc040a6af0 tx=0x7fcc040aa610 comp rx=0 tx=0).stop 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 shutdown_connections 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55886dc37e50 0x55886dc3a270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55886da30e40 0x55886dbd42f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55886dbb4bf0 0x55886dc36330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcbfe000700 0 client.26849 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 --2- 172.21.15.50:0/3336252572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886dbaec50 0x55886dbd4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.433+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 >> 172.21.15.50:0/3336252572 conn(0x55886da6fa80 msgr2=0x55886dbb2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.434+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 shutdown_connections 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.434+0000 7fcc29517b80 1 -- 172.21.15.50:0/3336252572 wait complete. 2024-05-10T21:17:37.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.434+0000 7fcc29517b80 20 client.26849 trim_cache size 0 max 16384 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": { 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "1fe31dca-6c29-4c80-8d3f-5a2cf3d85748": { 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/", 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tag": "1fe31dca-6c29-4c80-8d3f-5a2cf3d85748", 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "options": "recursive,force" 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:17:37.433 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 msgr2=0x7fcbf80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 0x7fcbf80b4e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009680 tx=0x7fcc00009260 comp rx=0 tx=0).stop 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 msgr2=0x7fcc1419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 0x7fcc1419adf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcc040098e0 tx=0x7fcc040057c0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 shutdown_connections 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcbf80b2900 0x7fcbf80b4e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc1410e8e0 0x7fcc1419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc14105500 0x7fcc1419adf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 --2- 172.21.15.50:0/3546716965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc14105e50 0x7fcc1419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.435+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 >> 172.21.15.50:0/3546716965 conn(0x7fcc140fedc0 msgr2=0x7fcc140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.436+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 shutdown_connections 2024-05-10T21:17:37.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:17:37.436+0000 7fcc1aa00700 1 -- 172.21.15.50:0/3546716965 wait complete. 2024-05-10T21:17:37.473 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.207+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml auth v1434 ap=1 s=4008 n(v0 rc2024-05-10T21:15:22.424414+0000 b4008 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff482be680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0f:[<0x20000000adc/comment-37.xml v1397>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0f:[<0x20000000adc/comment-37.xml v1434>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.473 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.207+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d29 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml auth v1440 ap=1 s=9329 n(v0 rc2024-05-10T21:15:23.626384+0000 b9329 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4953c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d29:[<0x20000000adc/comment-16.xml v1409>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d29:[<0x20000000adc/comment-16.xml v1440>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.473 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.211+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d7a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml auth v1419 ap=1 s=10964 n(v0 rc2024-05-10T21:15:33.883130+0000 b10964 1=1+0) | replicated=0 authpin=1 scrubqueue=0 0x55ff49df7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d7a:[<0x20000000adc/article-62.xml v1419>,<0x10000000399/blog-27 v99389>,<0x10000000208/blogtest_in v1584>,<0x10000000201/src v1774>,<0x10000000006/blogbench-1.0 v1555>,<0x10000000005/tmp v122>,<0x100000001fb/client.0 v123>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v135>,<0x10000000001/sv_1 v159>,<0x10000000000/qa v152>,<0x1/volumes v141>]//[]","memoryvalue":"(3)0x10000000d7a:[<0x20000000adc/article-62.xml v1419>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.212+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d12 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml auth v1432 ap=1 s=2556 n(v0 rc2024-05-10T21:15:22.546411+0000 b2556 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48068680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d12:[<0x20000000adc/comment-34.xml v1395>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d12:[<0x20000000adc/comment-34.xml v1432>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.213+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000cfd [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml auth v1438 ap=1 s=9225 n(v0 rc2024-05-10T21:15:22.067423+0000 b9225 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff49df5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000cfd:[<0x20000000adc/comment-55.xml v1403>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000cfd:[<0x20000000adc/comment-55.xml v1438>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.214+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml auth v1430 ap=1 s=4514 n(v0 rc2024-05-10T21:15:22.768405+0000 b4514 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47815b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1c:[<0x20000000adc/comment-25.xml v1405>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1c:[<0x20000000adc/comment-25.xml v1430>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.215+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d0b [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml auth v1436 ap=1 s=4656 n(v0 rc2024-05-10T21:15:22.345416+0000 b4656 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff496d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d0b:[<0x20000000adc/comment-41.xml v1401>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d0b:[<0x20000000adc/comment-41.xml v1436>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.221+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1e [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml auth v1426 ap=1 s=7174 n(v0 rc2024-05-10T21:15:22.790405+0000 b7174 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fb3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1e:[<0x20000000adc/comment-23.xml v1407>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1e:[<0x20000000adc/comment-23.xml v1426>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.474 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.222+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d1d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml auth v1428 ap=1 s=9814 n(v0 rc2024-05-10T21:15:22.780405+0000 b9814 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fb2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d1d:[<0x20000000adc/comment-24.xml v1393>,<0x10000000399/blog-27 v92878>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x10000000d1d:[<0x20000000adc/comment-24.xml v1428>,<0x10000000399/blog-27 v129841>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.737 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.642+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021bb [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml auth v980 ap=1 s=3480 n(v0 rc2024-05-10T21:13:40.062960+0000 b3480 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4840d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021bb:[<0x2000000216c/article-59.xml v980>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021bb:[<0x2000000216c/article-59.xml v980>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.738 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.642+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002355 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg auth v1014 ap=1 s=34057 n(v0 rc2024-05-10T21:13:43.912864+0000 b34057 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4840ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002355:[<0x2000000216c/picture-13.jpg v1014>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002355:[<0x2000000216c/picture-13.jpg v1014>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.738 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.643+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002346 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg auth v1018 ap=1 s=17911 n(v0 rc2024-05-10T21:13:43.768868+0000 b17911 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4840f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002346:[<0x2000000216c/picture-18.jpg v1018>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002346:[<0x2000000216c/picture-18.jpg v1018>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.738 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.644+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002189 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml auth v982 ap=1 s=6348 n(v0 rc2024-05-10T21:13:39.304979+0000 b6348 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4840c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002189:[<0x2000000216c/article-74.xml v982>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002189:[<0x2000000216c/article-74.xml v982>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.738 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.645+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002240 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml auth v984 ap=1 s=23165 n(v0 rc2024-05-10T21:13:41.050936+0000 b23165 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4840f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002240:[<0x2000000216c/article-18.xml v984>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002240:[<0x2000000216c/article-18.xml v984>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.738 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.646+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002343 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg auth v1012 ap=1 s=22045 n(v0 rc2024-05-10T21:13:43.748869+0000 b22045 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a2e0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002343:[<0x2000000216c/picture-19.jpg v1012>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002343:[<0x2000000216c/picture-19.jpg v1012>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.647+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021e5 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml auth v986 ap=1 s=17511 n(v0 rc2024-05-10T21:13:40.613947+0000 b17511 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4a2e2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021e5:[<0x2000000216c/article-51.xml v986>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021e5:[<0x2000000216c/article-51.xml v986>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.649+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002225 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml auth v988 ap=1 s=28517 n(v0 rc2024-05-10T21:13:40.934939+0000 b28517 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48392100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002225:[<0x2000000216c/article-27.xml v988>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002225:[<0x2000000216c/article-27.xml v988>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.651+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023a0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg auth v1074 ap=1 s=40241 n(v0 rc2024-05-10T21:15:19.104496+0000 b40241 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff48390580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023a0:[<0x2000000216c/picture-4.jpg v1061>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023a0:[<0x2000000216c/picture-4.jpg v1074>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.651+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002314 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg auth v990 ap=1 s=59948 n(v0 rc2024-05-10T21:13:42.019912+0000 b59948 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff48390000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002314:[<0x2000000216c/picture-35.jpg v990>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002314:[<0x2000000216c/picture-35.jpg v990>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.653+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023a9 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg auth v1072 ap=1 s=47083 n(v0 rc2024-05-10T21:15:19.152495+0000 b47083 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47a5f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023a9:[<0x2000000216c/picture-2.jpg v1055>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023a9:[<0x2000000216c/picture-2.jpg v1072>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.739 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.654+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002273 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg auth v992 ap=1 s=55681 n(v0 rc2024-05-10T21:13:41.248931+0000 b55681 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47a5e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002273:[<0x2000000216c/picture-78.jpg v992>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002273:[<0x2000000216c/picture-78.jpg v992>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.655+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002192 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml auth v994 ap=1 s=6195 n(v0 rc2024-05-10T21:13:39.352978+0000 b6195 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47a5e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002192:[<0x2000000216c/article-70.xml v994>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002192:[<0x2000000216c/article-70.xml v994>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.656+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002239 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml auth v996 ap=1 s=25207 n(v0 rc2024-05-10T21:13:41.022937+0000 b25207 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47a5ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002239:[<0x2000000216c/article-20.xml v996>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002239:[<0x2000000216c/article-20.xml v996>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.657+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002352 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg auth v1016 ap=1 s=21392 n(v0 rc2024-05-10T21:13:43.886865+0000 b21392 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff488e2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002352:[<0x2000000216c/picture-14.jpg v1016>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002352:[<0x2000000216c/picture-14.jpg v1016>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.658+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002173 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml auth v998 ap=1 s=25466 n(v0 rc2024-05-10T21:13:39.146983+0000 b25466 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff488e4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002173:[<0x2000000216c/article-83.xml v998>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002173:[<0x2000000216c/article-83.xml v998>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.659+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021b0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml auth v1000 ap=1 s=4307 n(v0 rc2024-05-10T21:13:39.790967+0000 b4307 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4989cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021b0:[<0x2000000216c/article-61.xml v1000>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021b0:[<0x2000000216c/article-61.xml v1000>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.660+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021f7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml auth v1004 ap=1 s=8564 n(v0 rc2024-05-10T21:13:40.719944+0000 b8564 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4989db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021f7:[<0x2000000216c/article-45.xml v1004>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021f7:[<0x2000000216c/article-45.xml v1004>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.740 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.660+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002201 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml auth v1002 ap=1 s=16580 n(v0 rc2024-05-10T21:13:40.770943+0000 b16580 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff4989d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002201:[<0x2000000216c/article-41.xml v1002>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002201:[<0x2000000216c/article-41.xml v1002>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.660+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002194 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml auth v1006 ap=1 s=8883 n(v0 rc2024-05-10T21:13:39.361978+0000 b8883 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47fa9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002194:[<0x2000000216c/article-69.xml v1006>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002194:[<0x2000000216c/article-69.xml v1006>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.661+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022d4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg auth v1008 ap=1 s=51934 n(v0 rc2024-05-10T21:13:41.742919+0000 b51934 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47fa8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022d4:[<0x2000000216c/picture-61.jpg v1008>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022d4:[<0x2000000216c/picture-61.jpg v1008>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.662+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002394 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg auth v1076 ap=1 s=12792 n(v0 rc2024-05-10T21:15:18.976499+0000 b12792 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002394:[<0x2000000216c/picture-7.jpg v1057>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002394:[<0x2000000216c/picture-7.jpg v1076>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.662+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002217 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml auth v1010 ap=1 s=3992 n(v0 rc2024-05-10T21:13:40.883940+0000 b3992 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47fa8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002217:[<0x2000000216c/article-31.xml v1010>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002217:[<0x2000000216c/article-31.xml v1010>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.664+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000238a [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg auth v1078 ap=1 s=43709 n(v0 rc2024-05-10T21:15:18.912501+0000 b43709 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff47fa7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000238a:[<0x2000000216c/picture-9.jpg v1059>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000238a:[<0x2000000216c/picture-9.jpg v1078>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.740+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000023a4 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg auth v1014 ap=1 s=25517 n(v0 rc2024-05-10T21:15:19.129496+0000 b25517 1=1+0) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ff4955ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000023a4:[<0x2000000216c/picture-3.jpg v1009>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000023a4:[<0x2000000216c/picture-3.jpg v1014>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:37.741 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.743+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022e2 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg auth v967 ap=1 s=32625 n(v0 rc2024-05-10T21:13:41.801917+0000 b32625 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49559b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022e2:[<0x2000000216c/picture-55.jpg v967>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022e2:[<0x2000000216c/picture-55.jpg v967>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.066 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.745+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002373 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg auth v993 ap=1 s=22449 n(v0 rc2024-05-10T21:13:44.266856+0000 b22449 1=1+0) | ptrwaiter=0 request=0 lock=0 caps=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55ff497f0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002373:[<0x2000000216c/picture-11.jpg v993>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002373:[<0x2000000216c/picture-11.jpg v993>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.066 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.746+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022a0 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg auth v969 ap=1 s=19808 n(v0 rc2024-05-10T21:13:41.488925+0000 b19808 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49559600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022a0:[<0x2000000216c/picture-70.jpg v969>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022a0:[<0x2000000216c/picture-70.jpg v969>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.066 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.747+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002265 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml auth v971 ap=1 s=24235 n(v0 rc2024-05-10T21:13:41.194932+0000 b24235 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49558580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002265:[<0x2000000216c/article-5.xml v971>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002265:[<0x2000000216c/article-5.xml v971>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.066 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.748+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002210 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml auth v973 ap=1 s=12263 n(v0 rc2024-05-10T21:13:40.840941+0000 b12263 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49932580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002210:[<0x2000000216c/article-34.xml v973>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002210:[<0x2000000216c/article-34.xml v973>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.067 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.748+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002178 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml auth v975 ap=1 s=13666 n(v0 rc2024-05-10T21:13:39.162983+0000 b13666 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49932b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002178:[<0x2000000216c/article-81.xml v975>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002178:[<0x2000000216c/article-81.xml v975>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.067 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.750+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000217f [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml auth v977 ap=1 s=3663 n(v0 rc2024-05-10T21:13:39.231981+0000 b3663 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49935180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000217f:[<0x2000000216c/article-78.xml v977>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000217f:[<0x2000000216c/article-78.xml v977>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.067 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.750+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000229c [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg auth v979 ap=1 s=57918 n(v0 rc2024-05-10T21:13:41.460926+0000 b57918 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49932000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000229c:[<0x2000000216c/picture-71.jpg v979>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000229c:[<0x2000000216c/picture-71.jpg v979>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.067 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.752+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000221d [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml auth v981 ap=1 s=5613 n(v0 rc2024-05-10T21:13:40.904939+0000 b5613 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff49934100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000221d:[<0x2000000216c/article-29.xml v981>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x2000000221d:[<0x2000000216c/article-29.xml v981>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.068 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.754+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021b7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml auth v983 ap=1 s=23667 n(v0 rc2024-05-10T21:13:39.957963+0000 b23667 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47ff3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021b7:[<0x2000000216c/article-60.xml v983>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021b7:[<0x2000000216c/article-60.xml v983>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.068 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.755+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000022c7 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg auth v985 ap=1 s=49765 n(v0 rc2024-05-10T21:13:41.687920+0000 b49765 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47ff2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000022c7:[<0x2000000216c/picture-63.jpg v985>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000022c7:[<0x2000000216c/picture-63.jpg v985>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.068 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.757+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002305 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg auth v987 ap=1 s=50193 n(v0 rc2024-05-10T21:13:41.959913+0000 b50193 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47ff5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002305:[<0x2000000216c/picture-40.jpg v987>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002305:[<0x2000000216c/picture-40.jpg v987>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.068 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.759+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000002234 [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml auth v989 ap=1 s=19847 n(v0 rc2024-05-10T21:13:41.003937+0000 b19847 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47ff5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000002234:[<0x2000000216c/article-22.xml v989>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x20000002234:[<0x2000000216c/article-22.xml v989>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.069 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:17:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:17:37.762+0000 7f8852000700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000021ea [2,head] /volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml auth v991 ap=1 s=28754 n(v0 rc2024-05-10T21:13:40.653946+0000 b28754 1=1+0) | importingcaps=0 caps=0 replicated=0 authpin=1 scrubqueue=0 0x55ff47ff4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000021ea:[<0x2000000216c/article-49.xml v991>,<0x10000000399/blog-67 v92892>,<0x10000000208/blogtest_in v1480>,<0x10000000201/src v1770>,<0x10000000006/blogbench-1.0 v1551>,<0x10000000005/tmp v118>,<0x100000001fb/client.0 v119>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v131>,<0x10000000001/sv_1 v155>,<0x10000000000/qa v148>,<0x1/volumes v137>]//[]","memoryvalue":"(3)0x200000021ea:[<0x2000000216c/article-49.xml v991>,<0x10000000399/blog-67 v129847>,<0x10000000208/blogtest_in v2303>,<0x10000000201/src v1811>,<0x10000000006/blogbench-1.0 v1575>,<0x10000000005/tmp v138>,<0x100000001fb/client.0 v139>,<0x100000001fa/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f v151>,<0x10000000001/sv_1 v135>,<0x10000000000/qa v128>,<0x1/volumes v167>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cfc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d38 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d18 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000da5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d23 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d05 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d06 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d39 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d3b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d08 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d00 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d32 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d09 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d30 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cff (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d11 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d02 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d15 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d3c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d14 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cfe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d20 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d03 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d2d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d29 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d12 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000cfd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d0b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.369 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:38 smithi092 ceph-mon[34415]: Scrub error on inode 0x10000000d1d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cfc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d38 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d18 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000da5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d23 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d05 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d06 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d39 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d3b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d08 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d00 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d32 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d09 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d30 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cff (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d11 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d02 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d15 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d3c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d14 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cfe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d20 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d03 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d2d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d29 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d12 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000cfd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d0b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:38 smithi142 ceph-mon[34846]: Scrub error on inode 0x10000000d1d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cfc (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d38 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d18 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000da5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d23 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d05 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d06 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d10 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d39 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d24 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d3b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d08 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d00 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d32 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d09 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d28 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d30 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d07 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cff (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d37 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d11 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d19 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d31 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d02 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d04 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d15 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d3c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d13 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d14 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cfe (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d20 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d3a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d25 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d03 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d2d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d16 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d01 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d29 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d7a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d12 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000cfd (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d0b (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1e (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:38 smithi050 ceph-mon[29546]: Scrub error on inode 0x10000000d1d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:38.606 INFO:tasks.cephfs.filesystem:scrub status for tag:1fe31dca-6c29-4c80-8d3f-5a2cf3d85748 - {'path': '/', 'tag': '1fe31dca-6c29-4c80-8d3f-5a2cf3d85748', 'options': 'recursive,force'} 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002355 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002346 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002189 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002240 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002343 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002225 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002314 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023a9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002273 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002192 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002239 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002352 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002173 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021b0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002201 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002194 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002394 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002217 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000238a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000023a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002373 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002265 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002210 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002178 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000217f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000229c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.367 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x2000000221d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000022c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002305 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x20000002234 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: Scrub error on inode 0x200000021ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:39.368 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:39 smithi092 ceph-mon[34415]: pgmap v725: 97 pgs: 97 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002355 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002346 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002189 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002240 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002343 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002225 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002314 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023a9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002273 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002192 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002239 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002352 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002173 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021b0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002201 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002194 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002394 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002217 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000238a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000023a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002373 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002265 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002210 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002178 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000217f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000229c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x2000000221d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000022c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002305 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x20000002234 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: Scrub error on inode 0x200000021ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:39 smithi142 ceph-mon[34846]: pgmap v725: 97 pgs: 97 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29542]: 2024-05-10T21:17:39.069+0000 7f28a6c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021bb (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002355 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002346 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002189 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002240 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002343 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021e5 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002225 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002314 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023a9 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002273 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002192 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002239 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002352 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002173 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021b0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021f7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002201 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002194 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022d4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002394 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002217 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000238a (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000023a4 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022e2 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002373 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022a0 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002265 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002210 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002178 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000217f (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000229c (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x2000000221d (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021b7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000022c7 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002305 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x20000002234 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: Scrub error on inode 0x200000021ea (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml) see mds.a log and `damage ls` output for details 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: scrub summary: idle+waiting paths [/] 2024-05-10T21:17:39.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:39 smithi050 ceph-mon[29546]: pgmap v725: 97 pgs: 97 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-05-10T21:17:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 -- 172.21.15.50:0/2906005229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0109850 msgr2=0x7f39e010bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 --2- 172.21.15.50:0/2906005229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0109850 0x7f39e010bc70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009ca0 tx=0x7f39d802a8c0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 -- 172.21.15.50:0/2906005229 shutdown_connections 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 --2- 172.21.15.50:0/2906005229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0106ef0 0x7f39e0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 --2- 172.21.15.50:0/2906005229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0109850 0x7f39e010bc70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 --2- 172.21.15.50:0/2906005229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0104590 0x7f39e01069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 -- 172.21.15.50:0/2906005229 >> 172.21.15.50:0/2906005229 conn(0x7f39e00fdb90 msgr2=0x7f39e0100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 -- 172.21.15.50:0/2906005229 shutdown_connections 2024-05-10T21:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.246+0000 7f39e5600640 1 -- 172.21.15.50:0/2906005229 wait complete. 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.247+0000 7f39e5600640 1 Processor -- start 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.247+0000 7f39e5600640 1 -- start start 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0109850 0x7f39e01025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010f620 con 0x7f39e0106ef0 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010f4a0 con 0x7f39e0104590 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010f7a0 con 0x7f39e0109850 2024-05-10T21:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39df400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39df400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46182/0 (socket says 172.21.15.50:46182) 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39df400640 1 -- 172.21.15.50:0/2725824971 learned_addr learned my addr 172.21.15.50:0/2725824971 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49908/0 (socket says 172.21.15.50:49908) 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.248+0000 7f39e4c00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0109850 0x7f39e01025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 -- 172.21.15.50:0/2725824971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0109850 msgr2=0x7f39e01025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0109850 0x7f39e01025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 -- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 msgr2=0x7f39e0102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 -- 172.21.15.50:0/2725824971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e0102cf0 con 0x7f39e0104590 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39df400640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dfe00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9c0 tx=0x7f39d000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dcc00640 1 -- 172.21.15.50:0/2725824971 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000ccc0 con 0x7f39e0104590 2024-05-10T21:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.249+0000 7f39dcc00640 1 -- 172.21.15.50:0/2725824971 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d00108e0 con 0x7f39e0104590 2024-05-10T21:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.250+0000 7f39dcc00640 1 -- 172.21.15.50:0/2725824971 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00187c0 con 0x7f39e0104590 2024-05-10T21:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.250+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01a4200 con 0x7f39e0104590 2024-05-10T21:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.250+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a4670 con 0x7f39e0104590 2024-05-10T21:17:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.253+0000 7f39edc00740 1 Processor -- start 2024-05-10T21:17:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.253+0000 7f39edc00740 1 -- start start 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512de4d0 0x559e516a8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e51605550 con 0x559e512de4d0 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559e515aabc0 con 0x559e512d62f0 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39edc00740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e51601510 con 0x559e512ff1b0 2024-05-10T21:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39df400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512de4d0 0x559e516a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50622/0 (socket says 172.21.15.50:50622) 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 -- 172.21.15.50:0/321509263 learned_addr learned my addr 172.21.15.50:0/321509263 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 -- 172.21.15.50:0/321509263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 msgr2=0x559e516a1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 -- 172.21.15.50:0/321509263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512de4d0 msgr2=0x559e516a8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512de4d0 0x559e516a8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.254+0000 7f39e4c00640 1 -- 172.21.15.50:0/321509263 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e51317210 con 0x559e512ff1b0 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.255+0000 7f39e4c00640 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39e006c2a0 tx=0x7f39d802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f578be1bbe53242 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.255+0000 7f39d5200640 1 -- 172.21.15.50:0/321509263 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8031650 con 0x559e512ff1b0 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.255+0000 7f39d5200640 1 -- 172.21.15.50:0/321509263 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d8036030 con 0x559e512ff1b0 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.255+0000 7f39d5200640 1 -- 172.21.15.50:0/321509263 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004650 con 0x559e512ff1b0 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.255+0000 7f39dcc00640 1 -- 172.21.15.50:0/2725824971 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f39d0010430 con 0x7f39e0104590 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 -- 172.21.15.50:0/321509263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 msgr2=0x559e516a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39e006c2a0 tx=0x7f39d802f3a0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 -- 172.21.15.50:0/321509263 shutdown_connections 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512de4d0 0x559e516a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 --2- 172.21.15.50:0/321509263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512ff1b0 0x559e516a1300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 -- 172.21.15.50:0/321509263 >> 172.21.15.50:0/321509263 conn(0x559e512f3300 msgr2=0x559e5169cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 -- 172.21.15.50:0/321509263 shutdown_connections 2024-05-10T21:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.256+0000 7f39edc00740 1 -- 172.21.15.50:0/321509263 wait complete. 2024-05-10T21:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.257+0000 7f39edc00740 1 -- start start 2024-05-10T21:17:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.257+0000 7f39edc00740 1 Processor -- start 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.257+0000 7f39dcc00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 0x7f39c40b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39dcc00640 1 -- 172.21.15.50:0/2725824971 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f39d00e24f0 con 0x7f39e0104590 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39e4c00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 0x7f39c40b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516aac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 0x559e516b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e516a8ff0 con 0x559e512d62f0 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559e515be660 con 0x559e512ff1b0 2024-05-10T21:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e516a9170 con 0x559e512de4d0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39edc00740 10 client.0 _ll_register_callbacks cb 0x559e51588400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39df400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39df400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50638/0 (socket says 172.21.15.50:50638) 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39df400640 1 -- 172.21.15.50:0/1078460617 learned_addr learned my addr 172.21.15.50:0/1078460617 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.258+0000 7f39dfe00640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39e4c00640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 0x559e516b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 -- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 msgr2=0x559e516b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 0x559e516b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 -- 172.21.15.50:0/1078460617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 msgr2=0x559e516aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 -- 172.21.15.50:0/1078460617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e513e3620 con 0x559e512de4d0 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39e4c00640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 0x559e516b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39df400640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f39cc012d20 tx=0x7f39cc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a48accd497185b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39c9600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39e4c00640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 0x7f39c40b4c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f39d802aeb0 tx=0x7f39d802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.259+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00fe20 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f39cc005020 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d8036030 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f39cc005220 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39edc00740 10 client.26806 fetch_fsmap learned FSMap version 40 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559e516b50f0 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d8036030 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.260+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f39cc01c470 con 0x559e512de4d0 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 10 client.26806 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 10 client.26806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 20 client.26806 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512bdd80 0x559e516b8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 4 client.26806 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e51613160 con 0x559e512bdd80 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.261+0000 7f39dfe00640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512bdd80 0x559e516b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.262+0000 7f39dfe00640 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512bdd80 0x559e516b8830 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c110d491eba73d15 server_cookie=bb4dcc86f42b91e in_seq=0 out_seq=0 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.262+0000 7f39c9600640 10 client.26806 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.262+0000 7f39c9600640 1 -- 172.21.15.50:0/1078460617 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e51613160 con 0x559e512bdd80 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.262+0000 7f39c9600640 10 client.26806 handle_command_reply: tid=0 2024-05-10T21:17:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 client.26806 shutdown 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 msgr2=0x559e516ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f39cc012d20 tx=0x7f39cc00f790 comp rx=0 tx=0).stop 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 shutdown_connections 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512bdd80 0x559e516b8830 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516ab190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512ff1b0 0x559e516b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 --2- 172.21.15.50:0/1078460617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39c9600640 0 client.26806 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 >> 172.21.15.50:0/1078460617 conn(0x559e512f3300 msgr2=0x559e5169bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 shutdown_connections 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 1 -- 172.21.15.50:0/1078460617 wait complete. 2024-05-10T21:17:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.263+0000 7f39edc00740 20 client.26806 trim_cache size 0 max 16384 2024-05-10T21:17:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 Processor -- start 2024-05-10T21:17:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 -- start start 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512f3300 0x559e5169cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01afb90 con 0x559e512f3300 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01ad3a0 con 0x559e512d62f0 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.281+0000 7f39edc00740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01ab1f0 con 0x559e512de4d0 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512f3300 0x559e5169cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49922/0 (socket says 172.21.15.50:49922) 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 -- 172.21.15.50:0/3016065232 learned_addr learned my addr 172.21.15.50:0/3016065232 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39e4c00640 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 -- 172.21.15.50:0/3016065232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 msgr2=0x559e516b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 -- 172.21.15.50:0/3016065232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512f3300 msgr2=0x559e5169cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512f3300 0x559e5169cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.282+0000 7f39df400640 1 -- 172.21.15.50:0/3016065232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e51697970 con 0x559e512d62f0 2024-05-10T21:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.283+0000 7f39df400640 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01c910 tx=0x7f39cc00fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e0b68ae45e3f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.283+0000 7f39cbe00640 1 -- 172.21.15.50:0/3016065232 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc000e60 con 0x559e512d62f0 2024-05-10T21:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.283+0000 7f39cbe00640 1 -- 172.21.15.50:0/3016065232 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc01cb80 con 0x559e512d62f0 2024-05-10T21:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.283+0000 7f39cbe00640 1 -- 172.21.15.50:0/3016065232 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004d90 con 0x559e512d62f0 2024-05-10T21:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 -- 172.21.15.50:0/3016065232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 msgr2=0x559e516b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01c910 tx=0x7f39cc00fd50 comp rx=0 tx=0).stop 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 -- 172.21.15.50:0/3016065232 shutdown_connections 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512de4d0 0x559e516b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512d62f0 0x559e516b2010 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 --2- 172.21.15.50:0/3016065232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512f3300 0x559e5169cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 -- 172.21.15.50:0/3016065232 >> 172.21.15.50:0/3016065232 conn(0x559e516a5a70 msgr2=0x559e5169c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 -- 172.21.15.50:0/3016065232 shutdown_connections 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.284+0000 7f39edc00740 1 -- 172.21.15.50:0/3016065232 wait complete. 2024-05-10T21:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 -- start start 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 Processor -- start 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 0x559e516b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01ab860 con 0x559e512d62f0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01accd0 con 0x559e512de4d0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01abf30 con 0x559e512f3300 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.285+0000 7f39edc00740 10 client.0 _ll_register_callbacks cb 0x7f39cc009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 0x559e516b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49928/0 (socket says 172.21.15.50:49928) 2024-05-10T21:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 -- 172.21.15.50:0/2404712846 learned_addr learned my addr 172.21.15.50:0/2404712846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46220/0 (socket says 172.21.15.50:46220) 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39e4c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 0x559e516b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50652/0 (socket says 172.21.15.50:50652) 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 -- 172.21.15.50:0/2404712846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 msgr2=0x559e516b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 0x559e516b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 -- 172.21.15.50:0/2404712846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 msgr2=0x559e516afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 -- 172.21.15.50:0/2404712846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e516b16d0 con 0x559e512de4d0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39df400640 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01c9b0 tx=0x7f39cc0129d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5f3a6e2c6f8c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.286+0000 7f39c9600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004680 con 0x559e512de4d0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39cc01cb80 con 0x559e512de4d0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f39cc004820 con 0x559e512de4d0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f39cc041050 con 0x559e512de4d0 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39edc00740 10 client.26861 fetch_fsmap learned FSMap version 40 2024-05-10T21:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559e516b2960 con 0x559e512de4d0 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.287+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc01cb80 con 0x559e512de4d0 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f39cc038070 con 0x559e512de4d0 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 10 client.26861 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 10 client.26861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 20 client.26861 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512ff1b0 0x559e517231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 4 client.26861 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e5160cfd0 con 0x559e512ff1b0 2024-05-10T21:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39dfe00640 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512ff1b0 0x559e517231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39dfe00640 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512ff1b0 0x559e517231b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb70e0f0f1bf261f server_cookie=ad2da454ea4bbe66 in_seq=0 out_seq=0 2024-05-10T21:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.288+0000 7f39c9600640 10 client.26861 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.299+0000 7f39c9600640 1 -- 172.21.15.50:0/2404712846 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e5160cfd0 con 0x559e512ff1b0 2024-05-10T21:17:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.299+0000 7f39c9600640 10 client.26861 handle_command_reply: tid=0 2024-05-10T21:17:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.300+0000 7f39edc00740 1 client.26861 shutdown 2024-05-10T21:17:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.300+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 msgr2=0x559e516b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.300+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01c9b0 tx=0x7f39cc0129d0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 shutdown_connections 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559e512ff1b0 0x559e517231b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e512f3300 0x559e516b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e512de4d0 0x559e516b01e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 --2- 172.21.15.50:0/2404712846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e512d62f0 0x559e516afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39c9600640 0 client.26861 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 >> 172.21.15.50:0/2404712846 conn(0x559e516a5a70 msgr2=0x559e516bcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 shutdown_connections 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 1 -- 172.21.15.50:0/2404712846 wait complete. 2024-05-10T21:17:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.301+0000 7f39edc00740 20 client.26861 trim_cache size 0 max 16384 2024-05-10T21:17:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:17:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:17:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:17:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 msgr2=0x7f39c40b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 0x7f39c40b4c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f39d802aeb0 tx=0x7f39d802a8c0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 msgr2=0x7f39e01039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9c0 tx=0x7f39d000ee80 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 shutdown_connections 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f39c40b2770 0x7f39c40b4c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39e0109850 0x7f39e01025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39e0106ef0 0x7f39e0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 --2- 172.21.15.50:0/2725824971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39e0104590 0x7f39e01039e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.302+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 >> 172.21.15.50:0/2725824971 conn(0x7f39e00fdb90 msgr2=0x7f39e0105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.303+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 shutdown_connections 2024-05-10T21:17:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:40.303+0000 7f39e5600640 1 -- 172.21.15.50:0/2725824971 wait complete. 2024-05-10T21:17:40.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:40 smithi092 ceph-mon[34415]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T21:17:40.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:40 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:17:40.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:40 smithi092 ceph-mon[34415]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:40.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:40 smithi092 ceph-mon[34415]: pgmap v726: 97 pgs: 97 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 421 op/s 2024-05-10T21:17:40.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:40 smithi142 ceph-mon[34846]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T21:17:40.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:40 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:17:40.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:40 smithi142 ceph-mon[34846]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:40.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:40 smithi142 ceph-mon[34846]: pgmap v726: 97 pgs: 97 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 421 op/s 2024-05-10T21:17:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:40 smithi050 ceph-mon[29546]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T21:17:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:40 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] up:active 2024-05-10T21:17:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:40 smithi050 ceph-mon[29546]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T21:17:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:40 smithi050 ceph-mon[29546]: pgmap v726: 97 pgs: 97 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 421 op/s 2024-05-10T21:17:43.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:42 smithi092 ceph-mon[34415]: pgmap v727: 97 pgs: 97 active+clean; 395 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 454 op/s 2024-05-10T21:17:43.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:42 smithi142 ceph-mon[34846]: pgmap v727: 97 pgs: 97 active+clean; 395 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 454 op/s 2024-05-10T21:17:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:42 smithi050 ceph-mon[29546]: pgmap v727: 97 pgs: 97 active+clean; 395 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 454 op/s 2024-05-10T21:17:45.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:44 smithi092 ceph-mon[34415]: pgmap v728: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 951 KiB/s wr, 418 op/s 2024-05-10T21:17:45.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:44 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:17:45.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:44 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:17:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:44 smithi142 ceph-mon[34846]: pgmap v728: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 951 KiB/s wr, 418 op/s 2024-05-10T21:17:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:44 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:17:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:44 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:17:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:44 smithi050 ceph-mon[29546]: pgmap v728: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 951 KiB/s wr, 418 op/s 2024-05-10T21:17:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:44 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:17:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:44 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:17:46.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:45 smithi092 ceph-mon[34415]: scrub summary: idle 2024-05-10T21:17:46.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:45 smithi142 ceph-mon[34846]: scrub summary: idle 2024-05-10T21:17:46.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:45 smithi050 ceph-mon[29546]: scrub summary: idle 2024-05-10T21:17:47.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:47 smithi092 ceph-mon[34415]: pgmap v729: 97 pgs: 97 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-05-10T21:17:47.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:47 smithi142 ceph-mon[34846]: pgmap v729: 97 pgs: 97 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-05-10T21:17:47.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:47 smithi050 ceph-mon[29546]: pgmap v729: 97 pgs: 97 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-05-10T21:17:49.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:49 smithi092 ceph-mon[34415]: pgmap v730: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-05-10T21:17:49.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:49 smithi142 ceph-mon[34846]: pgmap v730: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-05-10T21:17:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:49 smithi050 ceph-mon[29546]: pgmap v730: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-05-10T21:17:51.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:51 smithi092 ceph-mon[34415]: pgmap v731: 97 pgs: 97 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 989 KiB/s wr, 246 op/s 2024-05-10T21:17:51.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:51 smithi142 ceph-mon[34846]: pgmap v731: 97 pgs: 97 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 989 KiB/s wr, 246 op/s 2024-05-10T21:17:51.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:51 smithi050 ceph-mon[29546]: pgmap v731: 97 pgs: 97 active+clean; 374 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 989 KiB/s wr, 246 op/s 2024-05-10T21:17:53.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:53 smithi092 ceph-mon[34415]: pgmap v732: 97 pgs: 97 active+clean; 367 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 897 KiB/s wr, 253 op/s 2024-05-10T21:17:53.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:53 smithi142 ceph-mon[34846]: pgmap v732: 97 pgs: 97 active+clean; 367 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 897 KiB/s wr, 253 op/s 2024-05-10T21:17:53.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:53 smithi050 ceph-mon[29546]: pgmap v732: 97 pgs: 97 active+clean; 367 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 897 KiB/s wr, 253 op/s 2024-05-10T21:17:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 -- 172.21.15.50:0/351401592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4109820 msgr2=0x7fa5a410bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/351401592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4109820 0x7fa5a410bc40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009be0 tx=0x7fa59c02a6c0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 -- 172.21.15.50:0/351401592 shutdown_connections 2024-05-10T21:17:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/351401592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 0x7fa5a41092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/351401592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4109820 0x7fa5a410bc40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/351401592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4104560 0x7fa5a4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 -- 172.21.15.50:0/351401592 >> 172.21.15.50:0/351401592 conn(0x7fa5a40fdb60 msgr2=0x7fa5a4100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 -- 172.21.15.50:0/351401592 shutdown_connections 2024-05-10T21:17:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.573+0000 7fa5a8e00640 1 -- 172.21.15.50:0/351401592 wait complete. 2024-05-10T21:17:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.575+0000 7fa5a8e00640 1 Processor -- start 2024-05-10T21:17:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.575+0000 7fa5a8e00640 1 -- start start 2024-05-10T21:17:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.575+0000 7fa5a8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.575+0000 7fa5a8e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 0x7fa5a419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a8e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4109820 0x7fa5a41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410f580 con 0x7fa5a4104560 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a8e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410f400 con 0x7fa5a4109820 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a8e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a410f700 con 0x7fa5a4106ec0 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48486/0 (socket says 172.21.15.50:48486) 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 -- 172.21.15.50:0/735641383 learned_addr learned my addr 172.21.15.50:0/735641383 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 0x7fa5a419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4109820 0x7fa5a41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 -- 172.21.15.50:0/735641383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 msgr2=0x7fa5a419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 0x7fa5a419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 -- 172.21.15.50:0/735641383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4109820 msgr2=0x7fa5a41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.576+0000 7fa5a3400640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4109820 0x7fa5a41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa5a3400640 1 -- 172.21.15.50:0/735641383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41a1c90 con 0x7fa5a4104560 2024-05-10T21:17:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa5a3400640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9a0 tx=0x7fa59400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa59be00640 1 -- 172.21.15.50:0/735641383 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cd80 con 0x7fa5a4104560 2024-05-10T21:17:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa59be00640 1 -- 172.21.15.50:0/735641383 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5940035f0 con 0x7fa5a4104560 2024-05-10T21:17:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa59be00640 1 -- 172.21.15.50:0/735641383 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594010690 con 0x7fa5a4104560 2024-05-10T21:17:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.577+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41a1f80 con 0x7fa5a4104560 2024-05-10T21:17:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.578+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41a2390 con 0x7fa5a4104560 2024-05-10T21:17:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.582+0000 7fa59be00640 1 -- 172.21.15.50:0/735641383 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa594004000 con 0x7fa5a4104560 2024-05-10T21:17:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.583+0000 7fa5b1411740 1 Processor -- start 2024-05-10T21:17:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.583+0000 7fa5b1411740 1 -- start start 2024-05-10T21:17:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.583+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.583+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbd0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.583+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f80d4d0 0x55c96fbd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5b1411740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c96fb34550 con 0x55c96f80d4d0 2024-05-10T21:17:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5b1411740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c96fad9bc0 con 0x55c96f8052f0 2024-05-10T21:17:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5b1411740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c96fb30510 con 0x55c96f82e1b0 2024-05-10T21:17:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbd0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f80d4d0 0x55c96fbd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:38464/0 (socket says 172.21.15.50:38464) 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a3e00640 1 -- 172.21.15.50:0/3320612192 learned_addr learned my addr 172.21.15.50:0/3320612192 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.584+0000 7fa5a3e00640 1 -- 172.21.15.50:0/3320612192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 msgr2=0x55c96fbd0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbd0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3e00640 1 -- 172.21.15.50:0/3320612192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f80d4d0 msgr2=0x55c96fbd7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f80d4d0 0x55c96fbd7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3e00640 1 -- 172.21.15.50:0/3320612192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96f846210 con 0x55c96f82e1b0 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa59be00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 0x7fa5880b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009be0 tx=0x7fa59c005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c78e9dafaa105f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa59be00640 1 -- 172.21.15.50:0/735641383 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fa5940e24d0 con 0x7fa5a4104560 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa5a3400640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 0x7fa5880b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.585+0000 7fa598c00640 1 -- 172.21.15.50:0/3320612192 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c041030 con 0x55c96f82e1b0 2024-05-10T21:17:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.586+0000 7fa598c00640 1 -- 172.21.15.50:0/3320612192 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c030560 con 0x55c96f82e1b0 2024-05-10T21:17:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.586+0000 7fa598c00640 1 -- 172.21.15.50:0/3320612192 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c036040 con 0x55c96f82e1b0 2024-05-10T21:17:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 -- 172.21.15.50:0/3320612192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 msgr2=0x55c96fbd0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009be0 tx=0x7fa59c005c80 comp rx=0 tx=0).stop 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5a3400640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 0x7fa5880b4cf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa5940e2770 tx=0x7fa5940e5f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 -- 172.21.15.50:0/3320612192 shutdown_connections 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f80d4d0 0x55c96fbd7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 --2- 172.21.15.50:0/3320612192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f82e1b0 0x55c96fbd0300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 -- 172.21.15.50:0/3320612192 >> 172.21.15.50:0/3320612192 conn(0x55c96f822300 msgr2=0x55c96fbcbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 -- 172.21.15.50:0/3320612192 shutdown_connections 2024-05-10T21:17:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.587+0000 7fa5b1411740 1 -- 172.21.15.50:0/3320612192 wait complete. 2024-05-10T21:17:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.588+0000 7fa5b1411740 1 -- start start 2024-05-10T21:17:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.588+0000 7fa5b1411740 1 Processor -- start 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 0x55c96fbde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fbde7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f82e1b0 0x55c96fbe5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c96fbd8170 con 0x55c96f82e1b0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c96faed660 con 0x55c96f80d4d0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c96fbd82f0 con 0x55c96f8052f0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.589+0000 7fa5b1411740 10 client.0 _ll_register_callbacks cb 0x55c96fab7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 0x55c96fbde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fbde7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 0x55c96fbde270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:38466/0 (socket says 172.21.15.50:38466) 2024-05-10T21:17:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a3e00640 1 -- 172.21.15.50:0/2429298755 learned_addr learned my addr 172.21.15.50:0/2429298755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f82e1b0 0x55c96fbe5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 -- 172.21.15.50:0/2429298755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 msgr2=0x55c96fbde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 0x55c96fbde270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 -- 172.21.15.50:0/2429298755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f82e1b0 msgr2=0x55c96fbe5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f82e1b0 0x55c96fbe5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.590+0000 7fa5a2a00640 1 -- 172.21.15.50:0/2429298755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96f912620 con 0x55c96f80d4d0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fbde7b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5a419b1f0 tx=0x7fa590012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b527e2eb46a8e196 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa58d600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000f8c0 con 0x55c96f80d4d0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c030560 con 0x55c96f80d4d0 2024-05-10T21:17:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fa59000fa60 con 0x55c96f80d4d0 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa59000fc60 con 0x55c96f80d4d0 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa5b1411740 10 client.26873 fetch_fsmap learned FSMap version 40 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.591+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c96fbe8760 con 0x55c96f80d4d0 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c030560 con 0x55c96f80d4d0 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa590027420 con 0x55c96f80d4d0 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 10 client.26873 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 20 client.26873 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f7ecd80 0x55c96fbebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 4 client.26873 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:17:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.592+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96fb42160 con 0x55c96f7ecd80 2024-05-10T21:17:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.593+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f7ecd80 0x55c96fbebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.593+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f7ecd80 0x55c96fbebed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d334ad4101e5c0ec server_cookie=d495d7ca20c028f0 in_seq=0 out_seq=0 2024-05-10T21:17:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.593+0000 7fa58d600640 10 client.26873 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.594+0000 7fa58d600640 1 -- 172.21.15.50:0/2429298755 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c96fb42160 con 0x55c96f7ecd80 2024-05-10T21:17:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.594+0000 7fa58d600640 10 client.26873 handle_command_reply: tid=0 2024-05-10T21:17:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.594+0000 7fa5b1411740 1 client.26873 shutdown 2024-05-10T21:17:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 msgr2=0x55c96fbde7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fbde7b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5a419b1f0 tx=0x7fa590012b60 comp rx=0 tx=0).stop 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 shutdown_connections 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f7ecd80 0x55c96fbebed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa58d600640 0 client.26873 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f82e1b0 0x55c96fbe5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fbde7b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 --2- 172.21.15.50:0/2429298755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f8052f0 0x55c96fbde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 >> 172.21.15.50:0/2429298755 conn(0x55c96f822300 msgr2=0x55c96fbceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 shutdown_connections 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.595+0000 7fa5b1411740 1 -- 172.21.15.50:0/2429298755 wait complete. 2024-05-10T21:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.596+0000 7fa5b1411740 20 client.26873 trim_cache size 0 max 16384 2024-05-10T21:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.629+0000 7fa5b1411740 1 Processor -- start 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.629+0000 7fa5b1411740 1 -- start start 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 0x55c96fbcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f80d4d0 0x55c96fbca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41ab800 con 0x55c96f7ecd80 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a9010 con 0x55c96f8052f0 2024-05-10T21:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5b1411740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a6e60 con 0x55c96f80d4d0 2024-05-10T21:17:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 0x55c96fbcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5a2a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 0x55c96fbcbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48506/0 (socket says 172.21.15.50:48506) 2024-05-10T21:17:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.630+0000 7fa5a3e00640 1 -- 172.21.15.50:0/1481189270 learned_addr learned my addr 172.21.15.50:0/1481189270 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f80d4d0 0x55c96fbca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 -- 172.21.15.50:0/1481189270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f80d4d0 msgr2=0x55c96fbca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f80d4d0 0x55c96fbca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 -- 172.21.15.50:0/1481189270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 msgr2=0x55c96fbcbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 0x55c96fbcbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 -- 172.21.15.50:0/1481189270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96fbc5ef0 con 0x55c96f8052f0 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.631+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbca250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55c96fbcc8e0 tx=0x7fa590009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b0ab3af4b471e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.632+0000 7fa58fe00640 1 -- 172.21.15.50:0/1481189270 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590027420 con 0x55c96f8052f0 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.632+0000 7fa58fe00640 1 -- 172.21.15.50:0/1481189270 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5900275c0 con 0x55c96f8052f0 2024-05-10T21:17:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.632+0000 7fa58fe00640 1 -- 172.21.15.50:0/1481189270 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590005600 con 0x55c96f8052f0 2024-05-10T21:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 -- 172.21.15.50:0/1481189270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 msgr2=0x55c96fbca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbca250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55c96fbcc8e0 tx=0x7fa590009ec0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 -- 172.21.15.50:0/1481189270 shutdown_connections 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f80d4d0 0x55c96fbca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f8052f0 0x55c96fbca250 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 --2- 172.21.15.50:0/1481189270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f7ecd80 0x55c96fbcbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 -- 172.21.15.50:0/1481189270 >> 172.21.15.50:0/1481189270 conn(0x55c96f822300 msgr2=0x55c96fbcb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 -- 172.21.15.50:0/1481189270 shutdown_connections 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.633+0000 7fa5b1411740 1 -- 172.21.15.50:0/1481189270 wait complete. 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.634+0000 7fa5b1411740 1 -- start start 2024-05-10T21:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.634+0000 7fa5b1411740 1 Processor -- start 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 0x55c96fbe1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 0x55c96fbe3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fc56a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a74d0 con 0x55c96f8052f0 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a8940 con 0x55c96f80d4d0 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41a7ba0 con 0x55c96f7ecd80 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5b1411740 10 client.0 _ll_register_callbacks cb 0x7fa590009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 0x55c96fbe1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 0x55c96fbe1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:38486/0 (socket says 172.21.15.50:38486) 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5a3e00640 1 -- 172.21.15.50:0/289427640 learned_addr learned my addr 172.21.15.50:0/289427640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 0x55c96fbe3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.635+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fc56a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 -- 172.21.15.50:0/289427640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 msgr2=0x55c96fbe1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 0x55c96fbe1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 -- 172.21.15.50:0/289427640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 msgr2=0x55c96fbe3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 0x55c96fbe3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 -- 172.21.15.50:0/289427640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96fbe6120 con 0x55c96f80d4d0 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 0x55c96fbe3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5a3e00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fc56a60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0306e0 tx=0x7fa59c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d911e758e8af928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa58d600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c036040 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fa59c054c70 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.636+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5900275c0 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa59c041040 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 10 client.26879 fetch_fsmap learned FSMap version 40 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c96fc57f30 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5900275c0 con 0x55c96f80d4d0 2024-05-10T21:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa59c004620 con 0x55c96f80d4d0 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 10 client.26879 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 10 client.26879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 20 client.26879 populate_metadata read hostname 'smithi050' 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.637+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f82e1b0 0x55c96fc5a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.638+0000 7fa5b1411740 4 client.26879 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.638+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c96fb3bfd0 con 0x55c96f82e1b0 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.638+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f82e1b0 0x55c96fc5a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.638+0000 7fa5a2a00640 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f82e1b0 0x55c96fc5a510 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8888e08c027ae410 server_cookie=259d719e70a79d33 in_seq=0 out_seq=0 2024-05-10T21:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.638+0000 7fa58d600640 10 client.26879 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.646+0000 7fa58d600640 1 -- 172.21.15.50:0/289427640 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c96fb3bfd0 con 0x55c96f82e1b0 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.646+0000 7fa58d600640 10 client.26879 handle_command_reply: tid=0 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.646+0000 7fa5b1411740 1 client.26879 shutdown 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.646+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 msgr2=0x55c96fc56a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.646+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fc56a60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0306e0 tx=0x7fa59c02a6c0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 shutdown_connections 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55c96f82e1b0 0x55c96fc5a510 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c96f80d4d0 0x55c96fc56a60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c96f8052f0 0x55c96fbe3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 --2- 172.21.15.50:0/289427640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c96f7ecd80 0x55c96fbe1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa58d600640 0 client.26879 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 >> 172.21.15.50:0/289427640 conn(0x55c96f822300 msgr2=0x55c96fbf06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 shutdown_connections 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 1 -- 172.21.15.50:0/289427640 wait complete. 2024-05-10T21:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.647+0000 7fa5b1411740 20 client.26879 trim_cache size 0 max 16384 2024-05-10T21:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 msgr2=0x7fa5880b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 0x7fa5880b4cf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa5940e2770 tx=0x7fa5940e5f10 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 msgr2=0x7fa5a419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9a0 tx=0x7fa59400ee60 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 shutdown_connections 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fa5880b27e0 0x7fa5880b4cf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4106ec0 0x7fa5a419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5a4109820 0x7fa5a41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 --2- 172.21.15.50:0/735641383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5a4104560 0x7fa5a419a150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 >> 172.21.15.50:0/735641383 conn(0x7fa5a40fdb60 msgr2=0x7fa5a4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 shutdown_connections 2024-05-10T21:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:17:53.648+0000 7fa5a8e00640 1 -- 172.21.15.50:0/735641383 wait complete. 2024-05-10T21:17:55.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:55 smithi092 ceph-mon[34415]: pgmap v733: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 811 KiB/s wr, 224 op/s 2024-05-10T21:17:55.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:55 smithi142 ceph-mon[34846]: pgmap v733: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 811 KiB/s wr, 224 op/s 2024-05-10T21:17:55.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:55 smithi050 ceph-mon[29546]: pgmap v733: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 811 KiB/s wr, 224 op/s 2024-05-10T21:17:57.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:57 smithi092 ceph-mon[34415]: pgmap v734: 97 pgs: 97 active+clean; 354 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2024-05-10T21:17:57.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:57 smithi142 ceph-mon[34846]: pgmap v734: 97 pgs: 97 active+clean; 354 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2024-05-10T21:17:57.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:57 smithi050 ceph-mon[29546]: pgmap v734: 97 pgs: 97 active+clean; 354 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2024-05-10T21:17:59.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:17:59 smithi092 ceph-mon[34415]: pgmap v735: 97 pgs: 97 active+clean; 347 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 689 KiB/s wr, 220 op/s 2024-05-10T21:17:59.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:17:59 smithi142 ceph-mon[34846]: pgmap v735: 97 pgs: 97 active+clean; 347 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 689 KiB/s wr, 220 op/s 2024-05-10T21:17:59.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:17:59 smithi050 ceph-mon[29546]: pgmap v735: 97 pgs: 97 active+clean; 347 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 689 KiB/s wr, 220 op/s 2024-05-10T21:18:01.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:01 smithi092 ceph-mon[34415]: pgmap v736: 97 pgs: 97 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 728 KiB/s wr, 203 op/s 2024-05-10T21:18:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:01 smithi142 ceph-mon[34846]: pgmap v736: 97 pgs: 97 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 728 KiB/s wr, 203 op/s 2024-05-10T21:18:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:01 smithi050 ceph-mon[29546]: pgmap v736: 97 pgs: 97 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 728 KiB/s wr, 203 op/s 2024-05-10T21:18:03.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:03 smithi092 ceph-mon[34415]: pgmap v737: 97 pgs: 97 active+clean; 336 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 787 KiB/s wr, 240 op/s 2024-05-10T21:18:03.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:03 smithi142 ceph-mon[34846]: pgmap v737: 97 pgs: 97 active+clean; 336 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 787 KiB/s wr, 240 op/s 2024-05-10T21:18:03.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:03 smithi050 ceph-mon[29546]: pgmap v737: 97 pgs: 97 active+clean; 336 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 787 KiB/s wr, 240 op/s 2024-05-10T21:18:05.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:05 smithi092 ceph-mon[34415]: pgmap v738: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 723 KiB/s wr, 231 op/s 2024-05-10T21:18:05.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:05 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:18:05.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:05 smithi142 ceph-mon[34846]: pgmap v738: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 723 KiB/s wr, 231 op/s 2024-05-10T21:18:05.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:05 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:18:05.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:05 smithi050 ceph-mon[29546]: pgmap v738: 97 pgs: 97 active+clean; 328 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 723 KiB/s wr, 231 op/s 2024-05-10T21:18:05.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:05 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:18:06.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: pgmap v739: 97 pgs: 97 active+clean; 321 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:18:06.865 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:06 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: pgmap v739: 97 pgs: 97 active+clean; 321 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:18:06.875 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:06 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 -- 172.21.15.50:0/1284821902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd18103ab0 msgr2=0x7fdd181047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/1284821902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd18103ab0 0x7fdd181047b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009900 tx=0x7fdd0802a3f0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 -- 172.21.15.50:0/1284821902 shutdown_connections 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/1284821902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd181099f0 0x7fdd1810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/1284821902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/1284821902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd18103ab0 0x7fdd181047b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 -- 172.21.15.50:0/1284821902 >> 172.21.15.50:0/1284821902 conn(0x7fdd180fecc0 msgr2=0x7fdd18101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 -- 172.21.15.50:0/1284821902 shutdown_connections 2024-05-10T21:18:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.888+0000 7fdd1ce00640 1 -- 172.21.15.50:0/1284821902 wait complete. 2024-05-10T21:18:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.889+0000 7fdd1ce00640 1 Processor -- start 2024-05-10T21:18:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 -- start start 2024-05-10T21:18:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 0x7fdd18197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd181099f0 0x7fdd1819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1810d430 con 0x7fdd181099f0 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1810d2b0 con 0x7fdd18104cf0 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd1ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1810d5b0 con 0x7fdd18103ab0 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 0x7fdd18197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 0x7fdd18197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44740/0 (socket says 172.21.15.50:44740) 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd17400640 1 -- 172.21.15.50:0/3105086970 learned_addr learned my addr 172.21.15.50:0/3105086970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.890+0000 7fdd17e00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd181099f0 0x7fdd1819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 -- 172.21.15.50:0/3105086970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 msgr2=0x7fdd18197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 0x7fdd18197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 -- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd181099f0 msgr2=0x7fdd1819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd181099f0 0x7fdd1819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 -- 172.21.15.50:0/3105086970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1819fb10 con 0x7fdd18104cf0 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18198510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400bc70 tx=0x7fdd04008600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.891+0000 7fdd0fe00640 1 -- 172.21.15.50:0/3105086970 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400bca0 con 0x7fdd18104cf0 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.892+0000 7fdd0fe00640 1 -- 172.21.15.50:0/3105086970 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0400be40 con 0x7fdd18104cf0 2024-05-10T21:18:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.892+0000 7fdd0fe00640 1 -- 172.21.15.50:0/3105086970 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04014720 con 0x7fdd18104cf0 2024-05-10T21:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.892+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1819fe00 con 0x7fdd18104cf0 2024-05-10T21:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.892+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181a02c0 con 0x7fdd18104cf0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 Processor -- start 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 -- start start 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586471b0 0x55bf589e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd0fe00640 1 -- 172.21.15.50:0/3105086970 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdd04015400 con 0x7fdd18104cf0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf5894d550 con 0x55bf586264d0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf588f2bc0 con 0x55bf586471b0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.896+0000 7fdd253dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf58949510 con 0x55bf5861e2f0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44750/0 (socket says 172.21.15.50:44750) 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 -- 172.21.15.50:0/2844884172 learned_addr learned my addr 172.21.15.50:0/2844884172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17e00640 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586471b0 0x55bf589e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd16a00640 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 -- 172.21.15.50:0/2844884172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586471b0 msgr2=0x55bf589e9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586471b0 0x55bf589e9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 -- 172.21.15.50:0/2844884172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 msgr2=0x55bf589f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.897+0000 7fdd17400640 1 -- 172.21.15.50:0/2844884172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5865f210 con 0x55bf5861e2f0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.898+0000 7fdd17400640 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd0802a910 tx=0x7fdd08004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4b8de717785059 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.898+0000 7fdd0cc00640 1 -- 172.21.15.50:0/2844884172 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035030 con 0x55bf5861e2f0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.898+0000 7fdd0cc00640 1 -- 172.21.15.50:0/2844884172 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08040070 con 0x55bf5861e2f0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.898+0000 7fdd0cc00640 1 -- 172.21.15.50:0/2844884172 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080319c0 con 0x55bf5861e2f0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.899+0000 7fdd0fe00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 0x7fdcfc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.899+0000 7fdd0fe00640 1 -- 172.21.15.50:0/3105086970 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fdd040e23c0 con 0x7fdd18104cf0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.899+0000 7fdd17e00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 0x7fdcfc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.899+0000 7fdd253dc740 1 -- 172.21.15.50:0/2844884172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 msgr2=0x55bf589e9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.899+0000 7fdd253dc740 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd0802a910 tx=0x7fdd08004040 comp rx=0 tx=0).stop 2024-05-10T21:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 -- 172.21.15.50:0/2844884172 shutdown_connections 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e9880 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 --2- 172.21.15.50:0/2844884172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586471b0 0x55bf589e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 -- 172.21.15.50:0/2844884172 >> 172.21.15.50:0/2844884172 conn(0x55bf5863b300 msgr2=0x55bf589e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 -- 172.21.15.50:0/2844884172 shutdown_connections 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd17e00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 0x7fdcfc0b4d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55bf589ea4d0 tx=0x7fdd10015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 -- 172.21.15.50:0/2844884172 wait complete. 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.900+0000 7fdd253dc740 1 -- start start 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 Processor -- start 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf589f1170 con 0x55bf586471b0 2024-05-10T21:18:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf58906660 con 0x55bf5861e2f0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf589f12f0 con 0x55bf586264d0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd253dc740 10 client.0 _ll_register_callbacks cb 0x55bf588d0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd16a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd16a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44754/0 (socket says 172.21.15.50:44754) 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.901+0000 7fdd16a00640 1 -- 172.21.15.50:0/4038127570 learned_addr learned my addr 172.21.15.50:0/4038127570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd17400640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd17e00640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 -- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 msgr2=0x55bf589f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 -- 172.21.15.50:0/4038127570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 msgr2=0x55bf589fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 -- 172.21.15.50:0/4038127570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5872b620 con 0x55bf586264d0 2024-05-10T21:18:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd16a00640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd040a5a30 tx=0x7fdd04015c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16448397ff252a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd01600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a94c0 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fdd040e5c50 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.902+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08040070 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd040e5e50 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 10 client.26842 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bf589fd2a0 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08040070 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdd040aace0 con 0x55bf586264d0 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 10 client.26842 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 10 client.26842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 20 client.26842 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf58605d80 0x55bf58a00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 4 client.26842 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.903+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5895b160 con 0x55bf58605d80 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.904+0000 7fdd17400640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf58605d80 0x55bf58a00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.904+0000 7fdd17400640 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf58605d80 0x55bf58a00a00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93b5c58eb6e0e83 server_cookie=503d094c6d682e84 in_seq=0 out_seq=0 2024-05-10T21:18:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.904+0000 7fdd01600640 10 client.26842 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd01600640 1 -- 172.21.15.50:0/4038127570 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf5895b160 con 0x55bf58605d80 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd01600640 10 client.26842 handle_command_reply: tid=0 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 client.26842 shutdown 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 msgr2=0x55bf589f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd040a5a30 tx=0x7fdd04015c70 comp rx=0 tx=0).stop 2024-05-10T21:18:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 shutdown_connections 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf58605d80 0x55bf58a00a00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586264d0 0x55bf589f3340 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 --2- 172.21.15.50:0/4038127570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 >> 172.21.15.50:0/4038127570 conn(0x55bf5863b300 msgr2=0x55bf589e7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.905+0000 7fdd01600640 0 client.26842 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.906+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 shutdown_connections 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.906+0000 7fdd253dc740 1 -- 172.21.15.50:0/4038127570 wait complete. 2024-05-10T21:18:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.906+0000 7fdd253dc740 20 client.26842 trim_cache size 0 max 16384 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: pgmap v739: 97 pgs: 97 active+clean; 321 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-05-10T21:18:06.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:18:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:18:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:06 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:18:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 Processor -- start 2024-05-10T21:18:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 -- start start 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586264d0 0x55bf589fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181ab830 con 0x55bf586471b0 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181a9040 con 0x55bf586264d0 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.927+0000 7fdd253dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181a6eb0 con 0x55bf5861e2f0 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44758/0 (socket says 172.21.15.50:44758) 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 -- 172.21.15.50:0/130705872 learned_addr learned my addr 172.21.15.50:0/130705872 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17e00640 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd16a00640 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586264d0 0x55bf589fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 -- 172.21.15.50:0/130705872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586264d0 msgr2=0x55bf589fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586264d0 0x55bf589fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 -- 172.21.15.50:0/130705872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 msgr2=0x55bf589fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 -- 172.21.15.50:0/130705872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf589e04a0 con 0x55bf5861e2f0 2024-05-10T21:18:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.928+0000 7fdd17400640 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55bf589ea6e0 tx=0x7fdd08005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d1f7ce28810bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.929+0000 7fdd03e00640 1 -- 172.21.15.50:0/130705872 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08055070 con 0x55bf5861e2f0 2024-05-10T21:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.929+0000 7fdd03e00640 1 -- 172.21.15.50:0/130705872 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08040050 con 0x55bf5861e2f0 2024-05-10T21:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.929+0000 7fdd03e00640 1 -- 172.21.15.50:0/130705872 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035070 con 0x55bf5861e2f0 2024-05-10T21:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- 172.21.15.50:0/130705872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 msgr2=0x55bf589e4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55bf589ea6e0 tx=0x7fdd08005850 comp rx=0 tx=0).stop 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- 172.21.15.50:0/130705872 shutdown_connections 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586471b0 0x55bf589fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf586264d0 0x55bf589fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 --2- 172.21.15.50:0/130705872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf5861e2f0 0x55bf589e4b10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- 172.21.15.50:0/130705872 >> 172.21.15.50:0/130705872 conn(0x55bf5863b300 msgr2=0x55bf589e4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- 172.21.15.50:0/130705872 shutdown_connections 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- 172.21.15.50:0/130705872 wait complete. 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.930+0000 7fdd253dc740 1 -- start start 2024-05-10T21:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 Processor -- start 2024-05-10T21:18:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 0x55bf589f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181a7500 con 0x55bf586264d0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181a8970 con 0x55bf5861e2f0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181a7bd0 con 0x55bf586471b0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd253dc740 10 client.0 _ll_register_callbacks cb 0x7fdd04015d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.931+0000 7fdd17e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 0x55bf589f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 0x55bf589f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44770/0 (socket says 172.21.15.50:44770) 2024-05-10T21:18:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17e00640 1 -- 172.21.15.50:0/3656034191 learned_addr learned my addr 172.21.15.50:0/3656034191 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 -- 172.21.15.50:0/3656034191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 msgr2=0x55bf589f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 0x55bf589f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 -- 172.21.15.50:0/3656034191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 msgr2=0x55bf589f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 -- 172.21.15.50:0/3656034191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf589f97f0 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd17400640 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f7e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bf589f10d0 tx=0x7fdd0802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=869653f8d46ec4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.932+0000 7fdd01600640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035070 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fdd08031980 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08040050 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd08055040 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 10 client.26897 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bf589faa80 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08040050 con 0x55bf5861e2f0 2024-05-10T21:18:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdd0805a470 con 0x55bf5861e2f0 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 10 client.26897 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.933+0000 7fdd253dc740 20 client.26897 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf585db1c0 0x55bf58a6adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd253dc740 4 client.26897 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf58954fd0 con 0x55bf585db1c0 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf585db1c0 0x55bf58a6adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd16a00640 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf585db1c0 0x55bf58a6adc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf9471da5734705 server_cookie=59bc0c202a91858a in_seq=0 out_seq=0 2024-05-10T21:18:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.934+0000 7fdd01600640 10 client.26897 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.941+0000 7fdd01600640 1 -- 172.21.15.50:0/3656034191 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf58954fd0 con 0x55bf585db1c0 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.941+0000 7fdd01600640 10 client.26897 handle_command_reply: tid=0 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.941+0000 7fdd253dc740 1 client.26897 shutdown 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.941+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 msgr2=0x55bf589f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f7e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55bf589f10d0 tx=0x7fdd0802a3f0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 shutdown_connections 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55bf585db1c0 0x55bf58a6adc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf586471b0 0x55bf589f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf586264d0 0x55bf589f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd01600640 0 client.26897 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 --2- 172.21.15.50:0/3656034191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf5861e2f0 0x55bf589f7e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 >> 172.21.15.50:0/3656034191 conn(0x55bf5863b300 msgr2=0x55bf589e4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 shutdown_connections 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 1 -- 172.21.15.50:0/3656034191 wait complete. 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.942+0000 7fdd253dc740 20 client.26897 trim_cache size 0 max 16384 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 msgr2=0x7fdcfc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 0x7fdcfc0b4d70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55bf589ea4d0 tx=0x7fdd10015040 comp rx=0 tx=0).stop 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 msgr2=0x7fdd18198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18198510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400bc70 tx=0x7fdd04008600 comp rx=0 tx=0).stop 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 shutdown_connections 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fdcfc0b2860 0x7fdcfc0b4d70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd181099f0 0x7fdd1819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd18104cf0 0x7fdd18198510 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 --2- 172.21.15.50:0/3105086970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd18103ab0 0x7fdd18197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.943+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 >> 172.21.15.50:0/3105086970 conn(0x7fdd180fecc0 msgr2=0x7fdd181059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.944+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 shutdown_connections 2024-05-10T21:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:06.944+0000 7fdd1ce00640 1 -- 172.21.15.50:0/3105086970 wait complete. 2024-05-10T21:18:08.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T21:18:08.991 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:08 smithi050 ceph-mon[29546]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 932 KiB/s wr, 229 op/s 2024-05-10T21:18:09.093 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:09.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:08 smithi092 ceph-mon[34415]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 932 KiB/s wr, 229 op/s 2024-05-10T21:18:09.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:08 smithi142 ceph-mon[34846]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 932 KiB/s wr, 229 op/s 2024-05-10T21:18:11.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:10 smithi050 ceph-mon[29546]: pgmap v741: 97 pgs: 97 active+clean; 312 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-05-10T21:18:11.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:10 smithi092 ceph-mon[34415]: pgmap v741: 97 pgs: 97 active+clean; 312 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-05-10T21:18:11.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:10 smithi142 ceph-mon[34846]: pgmap v741: 97 pgs: 97 active+clean; 312 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 215 op/s 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 -- 172.21.15.50:0/3546268420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff74410af50 msgr2=0x7ff74410b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 --2- 172.21.15.50:0/3546268420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff74410af50 0x7ff74410b360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff730009990 tx=0x7ff730009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 -- 172.21.15.50:0/3546268420 shutdown_connections 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 --2- 172.21.15.50:0/3546268420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410b8a0 0x7ff744113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 --2- 172.21.15.50:0/3546268420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff744106620 0x7ff744106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 --2- 172.21.15.50:0/3546268420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff74410af50 0x7ff74410b360 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.707+0000 7ff749600700 1 -- 172.21.15.50:0/3546268420 >> 172.21.15.50:0/3546268420 conn(0x7ff744102030 msgr2=0x7ff744104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.708+0000 7ff749600700 1 -- 172.21.15.50:0/3546268420 shutdown_connections 2024-05-10T21:18:11.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.708+0000 7ff749600700 1 -- 172.21.15.50:0/3546268420 wait complete. 2024-05-10T21:18:11.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.709+0000 7ff749600700 1 Processor -- start 2024-05-10T21:18:11.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 -- start start 2024-05-10T21:18:11.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 0x7ff74419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff74410b8a0 0x7ff7441a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441164b0 con 0x7ff74410b8a0 2024-05-10T21:18:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff744116330 con 0x7ff744106620 2024-05-10T21:18:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff749600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff744116630 con 0x7ff74410af50 2024-05-10T21:18:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.710+0000 7ff743e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 0x7ff74419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57930/0 (socket says 172.21.15.50:57930) 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743e00700 1 -- 172.21.15.50:0/2135642994 learned_addr learned my addr 172.21.15.50:0/2135642994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff748c00700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff74410b8a0 0x7ff7441a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 -- 172.21.15.50:0/2135642994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 msgr2=0x7ff74419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 -- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff74410b8a0 msgr2=0x7ff7441a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff74410b8a0 0x7ff7441a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743400700 1 -- 172.21.15.50:0/2135642994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7441a1cf0 con 0x7ff74410af50 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.711+0000 7ff743e00700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:11.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff743400700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 0x7ff74419d3c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7300098e0 tx=0x7ff7300047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff740c00700 1 -- 172.21.15.50:0/2135642994 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730027070 con 0x7ff74410af50 2024-05-10T21:18:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff740c00700 1 -- 172.21.15.50:0/2135642994 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73000b610 con 0x7ff74410af50 2024-05-10T21:18:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff740c00700 1 -- 172.21.15.50:0/2135642994 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300173f0 con 0x7ff74410af50 2024-05-10T21:18:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7441a1f80 con 0x7ff74410af50 2024-05-10T21:18:11.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.712+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a23e0 con 0x7ff74410af50 2024-05-10T21:18:11.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.717+0000 7ff740c00700 1 -- 172.21.15.50:0/2135642994 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff73002a070 con 0x7ff74410af50 2024-05-10T21:18:11.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.718+0000 7ff759769b80 1 Processor -- start 2024-05-10T21:18:11.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.718+0000 7ff759769b80 1 -- start start 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559731ba4a20 con 0x559731c88a80 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559731b42760 con 0x559731b04e40 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff759769b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559731ba09e0 con 0x559731c82c40 2024-05-10T21:18:11.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff740c00700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 0x7ff7280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff748c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff748c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c88540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57942/0 (socket says 172.21.15.50:57942) 2024-05-10T21:18:11.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.719+0000 7ff748c00700 1 -- 172.21.15.50:0/245309699 learned_addr learned my addr 172.21.15.50:0/245309699 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:11.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743e00700 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff740c00700 1 -- 172.21.15.50:0/2135642994 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7ff7300e3150 con 0x7ff74410af50 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff748c00700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 0x7ff7280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 -- 172.21.15.50:0/245309699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 msgr2=0x559731c88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 -- 172.21.15.50:0/245309699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 msgr2=0x559731c82700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c82700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.720+0000 7ff743400700 1 -- 172.21.15.50:0/245309699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559731af18d0 con 0x559731c82c40 2024-05-10T21:18:11.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.721+0000 7ff743400700 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c83050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff7300a7a40 tx=0x7ff73009f0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933bc971a75fb1e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.721+0000 7ff739600700 1 -- 172.21.15.50:0/245309699 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300aa120 con 0x559731c82c40 2024-05-10T21:18:11.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.721+0000 7ff739600700 1 -- 172.21.15.50:0/245309699 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7300aa2e0 con 0x559731c82c40 2024-05-10T21:18:11.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.721+0000 7ff739600700 1 -- 172.21.15.50:0/245309699 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730013e80 con 0x559731c82c40 2024-05-10T21:18:11.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 -- 172.21.15.50:0/245309699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 msgr2=0x559731c83050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c83050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff7300a7a40 tx=0x7ff73009f0a0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff748c00700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 0x7ff7280b4d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x559731c839b0 tx=0x7ff73c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 -- 172.21.15.50:0/245309699 shutdown_connections 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c83050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 --2- 172.21.15.50:0/245309699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c82700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 -- 172.21.15.50:0/245309699 >> 172.21.15.50:0/245309699 conn(0x559731b49f70 msgr2=0x559731c7d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.722+0000 7ff759769b80 1 -- 172.21.15.50:0/245309699 shutdown_connections 2024-05-10T21:18:11.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.723+0000 7ff759769b80 1 -- 172.21.15.50:0/245309699 wait complete. 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.723+0000 7ff759769b80 1 -- start start 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 Processor -- start 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 0x559731c8c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559731c8f250 con 0x559731b04e40 2024-05-10T21:18:11.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559731b0ab20 con 0x559731c88a80 2024-05-10T21:18:11.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.724+0000 7ff759769b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559731c8f3d0 con 0x559731c82c40 2024-05-10T21:18:11.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff759769b80 10 client.0 _ll_register_callbacks cb 0x559731b4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:11.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff748c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff748c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57948/0 (socket says 172.21.15.50:57948) 2024-05-10T21:18:11.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff743e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 0x559731c8c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff743400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff743400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44808/0 (socket says 172.21.15.50:44808) 2024-05-10T21:18:11.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff743e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 0x559731c8c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58692/0 (socket says 172.21.15.50:58692) 2024-05-10T21:18:11.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.725+0000 7ff748c00700 1 -- 172.21.15.50:0/847947457 learned_addr learned my addr 172.21.15.50:0/847947457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 -- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 msgr2=0x559731c8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 -- 172.21.15.50:0/847947457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 msgr2=0x559731c8c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 0x559731c8c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 -- 172.21.15.50:0/847947457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559731b6a420 con 0x559731c82c40 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff748c00700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff743400700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7300a7a40 tx=0x7ff730005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57039ee3815bb81d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.726+0000 7ff72e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.727+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a9f40 con 0x559731c82c40 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.727+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7ff7300aac40 con 0x559731c82c40 2024-05-10T21:18:11.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.727+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7300aa2e0 con 0x559731c82c40 2024-05-10T21:18:11.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.727+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff7300aaed0 con 0x559731c82c40 2024-05-10T21:18:11.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.727+0000 7ff759769b80 10 client.26860 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:11.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.728+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559731c8f250 con 0x559731c82c40 2024-05-10T21:18:11.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.728+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7300aa2e0 con 0x559731c82c40 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.728+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff7300b3110 con 0x559731c82c40 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.728+0000 7ff759769b80 10 client.26860 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.728+0000 7ff759769b80 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.729+0000 7ff759769b80 20 client.26860 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.729+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731c9c130 0x559731c9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.729+0000 7ff759769b80 4 client.26860 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.729+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559731c4f1c0 con 0x559731c9c130 2024-05-10T21:18:11.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.729+0000 7ff743e00700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731c9c130 0x559731c9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.730+0000 7ff743e00700 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731c9c130 0x559731c9e570 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d159601b4012489 server_cookie=c904c4588bdd69c4 in_seq=0 out_seq=0 2024-05-10T21:18:11.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.730+0000 7ff72e000700 10 client.26860 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff72e000700 1 -- 172.21.15.50:0/847947457 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559731c4f1c0 con 0x559731c9c130 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff72e000700 10 client.26860 handle_command_reply: tid=0 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 client.26860 shutdown 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 msgr2=0x559731c8c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7300a7a40 tx=0x7ff730005ef0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 shutdown_connections 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731c9c130 0x559731c9e570 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff72e000700 0 client.26860 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c88a80 0x559731c8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731b04e40 0x559731c8c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 --2- 172.21.15.50:0/847947457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8c730 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 >> 172.21.15.50:0/847947457 conn(0x559731b49f70 msgr2=0x559731c816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 shutdown_connections 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.731+0000 7ff759769b80 1 -- 172.21.15.50:0/847947457 wait complete. 2024-05-10T21:18:11.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.732+0000 7ff759769b80 20 client.26860 trim_cache size 0 max 16384 2024-05-10T21:18:11.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.775+0000 7ff759769b80 1 Processor -- start 2024-05-10T21:18:11.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.775+0000 7ff759769b80 1 -- start start 2024-05-10T21:18:11.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b3370 con 0x559731c88a80 2024-05-10T21:18:11.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b0b40 con 0x559731b04e40 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff759769b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441aea10 con 0x559731c82c40 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff748c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff743e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff743e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:57952/0 (socket says 172.21.15.50:57952) 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff748c00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c8cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58704/0 (socket says 172.21.15.50:58704) 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff743e00700 1 -- 172.21.15.50:0/1873614154 learned_addr learned my addr 172.21.15.50:0/1873614154 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.776+0000 7ff743400700 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 -- 172.21.15.50:0/1873614154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 msgr2=0x559731c82700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 -- 172.21.15.50:0/1873614154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 msgr2=0x559731c8cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c8cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 -- 172.21.15.50:0/1873614154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559731c79c40 con 0x559731c82c40 2024-05-10T21:18:11.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743e00700 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff743400700 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8b1b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7300225b0 tx=0x7ff730022910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed8b5b3579f883c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.777+0000 7ff738c00700 1 -- 172.21.15.50:0/1873614154 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300aa120 con 0x559731c82c40 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.778+0000 7ff738c00700 1 -- 172.21.15.50:0/1873614154 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7300aa2e0 con 0x559731c82c40 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.778+0000 7ff738c00700 1 -- 172.21.15.50:0/1873614154 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300b3110 con 0x559731c82c40 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 -- 172.21.15.50:0/1873614154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 msgr2=0x559731c8b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8b1b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7300225b0 tx=0x7ff730022910 comp rx=0 tx=0).stop 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 -- 172.21.15.50:0/1873614154 shutdown_connections 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731b04e40 0x559731c82700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c8cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 --2- 172.21.15.50:0/1873614154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731c82c40 0x559731c8b1b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 -- 172.21.15.50:0/1873614154 >> 172.21.15.50:0/1873614154 conn(0x559731b49f70 msgr2=0x559731c864f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 -- 172.21.15.50:0/1873614154 shutdown_connections 2024-05-10T21:18:11.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.779+0000 7ff759769b80 1 -- 172.21.15.50:0/1873614154 wait complete. 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.780+0000 7ff759769b80 1 -- start start 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.780+0000 7ff759769b80 1 Processor -- start 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c9c130 0x559731c915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441af060 con 0x559731c88a80 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b0470 con 0x559731c9c130 2024-05-10T21:18:11.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441af6d0 con 0x559731b04e40 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.781+0000 7ff759769b80 10 client.0 _ll_register_callbacks cb 0x559731b4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44822/0 (socket says 172.21.15.50:44822) 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 -- 172.21.15.50:0/1742489756 learned_addr learned my addr 172.21.15.50:0/1742489756 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743400700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff748c00700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c9c130 0x559731c915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 -- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c9c130 msgr2=0x559731c915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c9c130 0x559731c915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 -- 172.21.15.50:0/1742489756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 msgr2=0x559731c94b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c94b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 -- 172.21.15.50:0/1742489756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559731c7ed40 con 0x559731b04e40 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743400700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff743e00700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559731c88010 tx=0x7ff734009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e491e5b63abc4533 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.782+0000 7ff72e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:11.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734020af0 con 0x559731b04e40 2024-05-10T21:18:11.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7300aa2e0 con 0x559731b04e40 2024-05-10T21:18:11.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7ff734020c90 con 0x559731b04e40 2024-05-10T21:18:11.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff734004830 con 0x559731b04e40 2024-05-10T21:18:11.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff759769b80 10 client.26866 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:11.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.783+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559731d06e20 con 0x559731b04e40 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7300aa2e0 con 0x559731b04e40 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff7340220b0 con 0x559731b04e40 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 20 client.26866 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731d07550 0x559731d09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 4 client.26866 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T21:18:11.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.784+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559731bb2400 con 0x559731d07550 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.785+0000 7ff743400700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731d07550 0x559731d09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.785+0000 7ff743400700 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731d07550 0x559731d09970 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7165b727d424f85 server_cookie=dfb3ccc5abc5f930 in_seq=0 out_seq=0 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.785+0000 7ff72e000700 10 client.26866 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff72e000700 1 -- 172.21.15.50:0/1742489756 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559731bb2400 con 0x559731d07550 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff72e000700 10 client.26866 handle_command_reply: tid=0 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff759769b80 1 client.26866 shutdown 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 msgr2=0x559731c94620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559731c88010 tx=0x7ff734009e50 comp rx=0 tx=0).stop 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 shutdown_connections 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.786+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x559731d07550 0x559731d09970 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559731c9c130 0x559731c915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559731b04e40 0x559731c94620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 --2- 172.21.15.50:0/1742489756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559731c88a80 0x559731c94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 >> 172.21.15.50:0/1742489756 conn(0x559731b49f70 msgr2=0x559731c772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff72e000700 0 client.26866 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 shutdown_connections 2024-05-10T21:18:11.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 1 -- 172.21.15.50:0/1742489756 wait complete. 2024-05-10T21:18:11.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.787+0000 7ff759769b80 20 client.26866 trim_cache size 0 max 16384 2024-05-10T21:18:11.785 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:11.785 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "no active scrubs running", 2024-05-10T21:18:11.785 INFO:teuthology.orchestra.run.smithi050.stdout: "scrubs": {} 2024-05-10T21:18:11.785 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 msgr2=0x7ff7280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 0x7ff7280b4d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x559731c839b0 tx=0x7ff73c004b40 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 msgr2=0x7ff74419d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 0x7ff74419d3c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7300098e0 tx=0x7ff7300047b0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 shutdown_connections 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff7280b2880 0x7ff7280b4d90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744106620 0x7ff74419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff74410b8a0 0x7ff7441a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 --2- 172.21.15.50:0/2135642994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff74410af50 0x7ff74419d3c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.788+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 >> 172.21.15.50:0/2135642994 conn(0x7ff744102030 msgr2=0x7ff74410a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.789+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 shutdown_connections 2024-05-10T21:18:11.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:11.790+0000 7ff749600700 1 -- 172.21.15.50:0/2135642994 wait complete. 2024-05-10T21:18:12.737 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T21:18:12.738 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:18:13.237 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:13.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:13 smithi092 ceph-mon[34415]: pgmap v742: 97 pgs: 97 active+clean; 299 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 256 op/s 2024-05-10T21:18:13.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:13 smithi142 ceph-mon[34846]: pgmap v742: 97 pgs: 97 active+clean; 299 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 256 op/s 2024-05-10T21:18:13.384 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:13 smithi050 ceph-mon[29546]: pgmap v742: 97 pgs: 97 active+clean; 299 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 256 op/s 2024-05-10T21:18:15.362 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:15 smithi050 ceph-mon[29546]: pgmap v743: 97 pgs: 97 active+clean; 293 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 868 KiB/s wr, 234 op/s 2024-05-10T21:18:15.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:15 smithi092 ceph-mon[34415]: pgmap v743: 97 pgs: 97 active+clean; 293 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 868 KiB/s wr, 234 op/s 2024-05-10T21:18:15.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:15 smithi142 ceph-mon[34846]: pgmap v743: 97 pgs: 97 active+clean; 293 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 868 KiB/s wr, 234 op/s 2024-05-10T21:18:15.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 -- 172.21.15.50:0/1950842207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 msgr2=0x7f0dd4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:15.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1950842207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd4104a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009990 tx=0x7f0dc4009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 -- 172.21.15.50:0/1950842207 shutdown_connections 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1950842207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd4104a00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1950842207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4109870 0x7f0dd41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1950842207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4108f20 0x7f0dd4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 -- 172.21.15.50:0/1950842207 >> 172.21.15.50:0/1950842207 conn(0x7f0dd4100040 msgr2=0x7f0dd4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 -- 172.21.15.50:0/1950842207 shutdown_connections 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.822+0000 7f0ddac00700 1 -- 172.21.15.50:0/1950842207 wait complete. 2024-05-10T21:18:15.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.823+0000 7f0ddac00700 1 Processor -- start 2024-05-10T21:18:15.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.824+0000 7f0ddac00700 1 -- start start 2024-05-10T21:18:15.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.824+0000 7f0ddac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:15.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.824+0000 7f0ddac00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4108f20 0x7f0dd419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.824+0000 7f0ddac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4109870 0x7f0dd419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.824+0000 7f0ddac00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd41144f0 con 0x7f0dd4108f20 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0ddac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4114370 con 0x7f0dd41045f0 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0ddac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4114670 con 0x7f0dd4109870 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dcb400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4108f20 0x7f0dd419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4109870 0x7f0dd419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55532/0 (socket says 172.21.15.50:55532) 2024-05-10T21:18:15.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 -- 172.21.15.50:0/1848326427 learned_addr learned my addr 172.21.15.50:0/1848326427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:15.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 -- 172.21.15.50:0/1848326427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4109870 msgr2=0x7f0dd419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:15.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4109870 0x7f0dd419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 -- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4108f20 msgr2=0x7f0dd419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:15.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4108f20 0x7f0dd419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.825+0000 7f0dd3e00700 1 -- 172.21.15.50:0/1848326427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd419fd50 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0dd3e00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4005910 tx=0x7f0dc40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4027070 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc400b610 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40173f0 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd419ffe0 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.826+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41a0470 con 0x7f0dd41045f0 2024-05-10T21:18:15.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.827+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd40512c0 con 0x7f0dd41045f0 2024-05-10T21:18:15.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.830+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0dc402a070 con 0x7f0dd41045f0 2024-05-10T21:18:15.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.831+0000 7f0dd1600700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 0x7f0db80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:15.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.832+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f0dc40e3180 con 0x7f0dd41045f0 2024-05-10T21:18:15.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.832+0000 7f0dcb400700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 0x7f0db80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:15.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.833+0000 7f0dcb400700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 0x7f0db80b4e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4103b30 tx=0x7f0dbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:15.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:15.833+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0dc4013070 con 0x7f0dd41045f0 2024-05-10T21:18:16.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.158+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd402e5f0 con 0x7f0dd41045f0 2024-05-10T21:18:16.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.159+0000 7f0dd1600700 1 -- 172.21.15.50:0/1848326427 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7f0dc40aaf00 con 0x7f0dd41045f0 2024-05-10T21:18:16.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:18:16.156 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:17:38.867551+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":206,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26813":{"gid":26813,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1239523786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1239523786},{"type":"v1","addr":"172.21.15.92:6837","nonce":1239523786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.163+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 msgr2=0x7f0db80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.163+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 0x7f0db80b4e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4103b30 tx=0x7f0dbc00b410 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.163+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 msgr2=0x7f0dd419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.163+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4005910 tx=0x7f0dc40047b0 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 shutdown_connections 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f0db80b2900 0x7f0db80b4e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dd41045f0 0x7f0dd419ae80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4109870 0x7f0dd419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 --2- 172.21.15.50:0/1848326427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd4108f20 0x7f0dd419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 >> 172.21.15.50:0/1848326427 conn(0x7f0dd4100040 msgr2=0x7f0dd41086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:16.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 shutdown_connections 2024-05-10T21:18:16.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:16.164+0000 7f0ddac00700 1 -- 172.21.15.50:0/1848326427 wait complete. 2024-05-10T21:18:16.161 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 40 2024-05-10T21:18:16.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T21:18:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:17 smithi050 ceph-mon[29546]: pgmap v744: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 272 op/s 2024-05-10T21:18:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:17 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1848326427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:17.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:17 smithi092 ceph-mon[34415]: pgmap v744: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 272 op/s 2024-05-10T21:18:17.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:17 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1848326427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:17.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:17 smithi142 ceph-mon[34846]: pgmap v744: 97 pgs: 97 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 272 op/s 2024-05-10T21:18:17.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:17 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1848326427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:17.438 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:19.100 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:19 smithi050 ceph-mon[29546]: pgmap v745: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 904 KiB/s wr, 283 op/s 2024-05-10T21:18:19.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:19 smithi092 ceph-mon[34415]: pgmap v745: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 904 KiB/s wr, 283 op/s 2024-05-10T21:18:19.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:19 smithi142 ceph-mon[34846]: pgmap v745: 97 pgs: 97 active+clean; 268 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 904 KiB/s wr, 283 op/s 2024-05-10T21:18:19.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 -- 172.21.15.50:0/1566640903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254076430 msgr2=0x7fc254110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 --2- 172.21.15.50:0/1566640903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254076430 0x7fc254110500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 -- 172.21.15.50:0/1566640903 shutdown_connections 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 --2- 172.21.15.50:0/1566640903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254075ae0 0x7fc254075ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 --2- 172.21.15.50:0/1566640903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254110a40 0x7fc254112e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 --2- 172.21.15.50:0/1566640903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254076430 0x7fc254110500 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 -- 172.21.15.50:0/1566640903 >> 172.21.15.50:0/1566640903 conn(0x7fc254079c20 msgr2=0x7fc25407a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.872+0000 7fc253200700 1 -- 172.21.15.50:0/1566640903 shutdown_connections 2024-05-10T21:18:19.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.873+0000 7fc253200700 1 -- 172.21.15.50:0/1566640903 wait complete. 2024-05-10T21:18:19.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.874+0000 7fc253200700 1 Processor -- start 2024-05-10T21:18:19.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.874+0000 7fc253200700 1 -- start start 2024-05-10T21:18:19.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254076430 0x7fc25419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254110a40 0x7fc2541a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541164f0 con 0x7fc254076430 2024-05-10T21:18:19.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc254116370 con 0x7fc254110a40 2024-05-10T21:18:19.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc253200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc254116670 con 0x7fc254075ae0 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc251e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc251e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55122/0 (socket says 172.21.15.50:55122) 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc251e00700 1 -- 172.21.15.50:0/2165558664 learned_addr learned my addr 172.21.15.50:0/2165558664 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc249400700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254076430 0x7fc25419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.875+0000 7fc252800700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254110a40 0x7fc2541a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 -- 172.21.15.50:0/2165558664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254110a40 msgr2=0x7fc2541a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254110a40 0x7fc2541a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 -- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254076430 msgr2=0x7fc25419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254076430 0x7fc25419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 -- 172.21.15.50:0/2165558664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2541a1d50 con 0x7fc254075ae0 2024-05-10T21:18:19.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc251e00700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c370 tx=0x7fc24400c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.876+0000 7fc24b400700 1 -- 172.21.15.50:0/2165558664 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440092e0 con 0x7fc254075ae0 2024-05-10T21:18:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.877+0000 7fc24b400700 1 -- 172.21.15.50:0/2165558664 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc24400cbe0 con 0x7fc254075ae0 2024-05-10T21:18:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.877+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541a1fe0 con 0x7fc254075ae0 2024-05-10T21:18:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.877+0000 7fc24b400700 1 -- 172.21.15.50:0/2165558664 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440046b0 con 0x7fc254075ae0 2024-05-10T21:18:19.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.877+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541a9960 con 0x7fc254075ae0 2024-05-10T21:18:19.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.880+0000 7fc24b400700 1 -- 172.21.15.50:0/2165558664 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc24401a020 con 0x7fc254075ae0 2024-05-10T21:18:19.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 Processor -- start 2024-05-10T21:18:19.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 -- start start 2024-05-10T21:18:19.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc24b400700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 0x7fc2380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc24b400700 1 -- 172.21.15.50:0/2165558664 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fc2440e1b00 con 0x7fc254075ae0 2024-05-10T21:18:19.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f3b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561829e5ca20 con 0x561829f3ac40 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561829dfa760 con 0x561829dbce40 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.882+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561829e589e0 con 0x561829f40a80 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc252800700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 0x7fc2380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc252800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f3b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55138/0 (socket says 172.21.15.50:55138) 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 -- 172.21.15.50:0/1831078475 learned_addr learned my addr 172.21.15.50:0/1831078475 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc251e00700 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 -- 172.21.15.50:0/1831078475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 msgr2=0x561829f40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 -- 172.21.15.50:0/1831078475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 msgr2=0x561829f3b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f3b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.883+0000 7fc249400700 1 -- 172.21.15.50:0/1831078475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561829da98d0 con 0x561829f40a80 2024-05-10T21:18:19.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.884+0000 7fc249400700 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188161584a36c7df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.884+0000 7fc243400700 1 -- 172.21.15.50:0/1831078475 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c027070 con 0x561829f40a80 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.884+0000 7fc243400700 1 -- 172.21.15.50:0/1831078475 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c00bbd0 con 0x561829f40a80 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.884+0000 7fc252800700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 0x7fc2380b4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097a0 tx=0x7fc24c009990 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.884+0000 7fc243400700 1 -- 172.21.15.50:0/1831078475 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c003d00 con 0x561829f40a80 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 -- 172.21.15.50:0/1831078475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 msgr2=0x561829f3a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c0096a0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 -- 172.21.15.50:0/1831078475 shutdown_connections 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f3b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/1831078475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f40a80 0x561829f3a700 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.885+0000 7fc2679a5b80 1 -- 172.21.15.50:0/1831078475 >> 172.21.15.50:0/1831078475 conn(0x561829e01f70 msgr2=0x561829f35400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.886+0000 7fc2679a5b80 1 -- 172.21.15.50:0/1831078475 shutdown_connections 2024-05-10T21:18:19.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.886+0000 7fc2679a5b80 1 -- 172.21.15.50:0/1831078475 wait complete. 2024-05-10T21:18:19.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.886+0000 7fc2679a5b80 1 -- start start 2024-05-10T21:18:19.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.887+0000 7fc2679a5b80 1 Processor -- start 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829f40a80 0x561829f44bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561829f47490 con 0x561829f3ac40 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561829dc2b20 con 0x561829f40a80 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561829f47610 con 0x561829dbce40 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc2679a5b80 10 client.0 _ll_register_callbacks cb 0x561829e03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55148/0 (socket says 172.21.15.50:55148) 2024-05-10T21:18:19.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 -- 172.21.15.50:0/889131978 learned_addr learned my addr 172.21.15.50:0/889131978 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:19.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc249400700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc252800700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829f40a80 0x561829f44bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 -- 172.21.15.50:0/889131978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829f40a80 msgr2=0x561829f44bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829f40a80 0x561829f44bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 -- 172.21.15.50:0/889131978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 msgr2=0x561829f446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.888+0000 7fc251e00700 1 -- 172.21.15.50:0/889131978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561829e22420 con 0x561829dbce40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc249400700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc251e00700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2440a9080 tx=0x7fc244009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c45aee8f4dc13a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc237e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440c3060 con 0x561829dbce40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fc244004850 con 0x561829dbce40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c00bbd0 con 0x561829dbce40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc2440ca050 con 0x561829dbce40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc2679a5b80 10 client.26884 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:19.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.889+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561829f51f90 con 0x561829dbce40 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c00bbd0 con 0x561829dbce40 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc2440a4a60 con 0x561829dbce40 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 10 client.26884 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 10 client.26884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 20 client.26884 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829f54520 0x561829f569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 4 client.26884 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561829f071c0 con 0x561829f54520 2024-05-10T21:18:19.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.890+0000 7fc249400700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829f54520 0x561829f569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.891+0000 7fc249400700 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829f54520 0x561829f569c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c368380cd0ecdf2 server_cookie=a65e9901ffa6f86c in_seq=0 out_seq=0 2024-05-10T21:18:19.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.891+0000 7fc237e00700 10 client.26884 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc237e00700 1 -- 172.21.15.50:0/889131978 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561829f071c0 con 0x561829f54520 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc237e00700 10 client.26884 handle_command_reply: tid=0 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 client.26884 shutdown 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 msgr2=0x561829f44170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2440a9080 tx=0x7fc244009d60 comp rx=0 tx=0).stop 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 shutdown_connections 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829f54520 0x561829f569c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829f40a80 0x561829f44bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829dbce40 0x561829f44170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/889131978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f3ac40 0x561829f446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc237e00700 0 client.26884 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.892+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 >> 172.21.15.50:0/889131978 conn(0x561829e01f70 msgr2=0x561829f396f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.893+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 shutdown_connections 2024-05-10T21:18:19.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.893+0000 7fc2679a5b80 1 -- 172.21.15.50:0/889131978 wait complete. 2024-05-10T21:18:19.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.893+0000 7fc2679a5b80 20 client.26884 trim_cache size 0 max 16384 2024-05-10T21:18:19.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.911+0000 7fc2679a5b80 1 Processor -- start 2024-05-10T21:18:19.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.911+0000 7fc2679a5b80 1 -- start start 2024-05-10T21:18:19.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.911+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f3a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f44a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541b34d0 con 0x561829f40a80 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541b0ca0 con 0x561829dbce40 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc2679a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541aeb10 con 0x561829f3ac40 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc249400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc249400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55162/0 (socket says 172.21.15.50:55162) 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc249400700 1 -- 172.21.15.50:0/3052347229 learned_addr learned my addr 172.21.15.50:0/3052347229 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc251e00700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f3a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc252800700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f44a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc251e00700 1 -- 172.21.15.50:0/3052347229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 msgr2=0x561829f354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.912+0000 7fc251e00700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc251e00700 1 -- 172.21.15.50:0/3052347229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 msgr2=0x561829f44a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc251e00700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f44a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc251e00700 1 -- 172.21.15.50:0/3052347229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561829f36d40 con 0x561829dbce40 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc249400700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc251e00700 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f3a700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc244000f80 tx=0x7fc244019fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8a2863f09afd5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc242a00700 1 -- 172.21.15.50:0/3052347229 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440ad4a0 con 0x561829dbce40 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc242a00700 1 -- 172.21.15.50:0/3052347229 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2440b1f50 con 0x561829dbce40 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.913+0000 7fc242a00700 1 -- 172.21.15.50:0/3052347229 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a7f30 con 0x561829dbce40 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 -- 172.21.15.50:0/3052347229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 msgr2=0x561829f3a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f3a700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc244000f80 tx=0x7fc244019fb0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 -- 172.21.15.50:0/3052347229 shutdown_connections 2024-05-10T21:18:19.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f3a700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/3052347229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.914+0000 7fc2679a5b80 1 -- 172.21.15.50:0/3052347229 >> 172.21.15.50:0/3052347229 conn(0x561829e01f70 msgr2=0x561829f3de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.915+0000 7fc2679a5b80 1 -- 172.21.15.50:0/3052347229 shutdown_connections 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.915+0000 7fc2679a5b80 1 -- 172.21.15.50:0/3052347229 wait complete. 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.915+0000 7fc2679a5b80 1 -- start start 2024-05-10T21:18:19.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.916+0000 7fc2679a5b80 1 Processor -- start 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f5b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc251e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc251e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55564/0 (socket says 172.21.15.50:55564) 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc251e00700 1 -- 172.21.15.50:0/4098367807 learned_addr learned my addr 172.21.15.50:0/4098367807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541af160 con 0x561829f40a80 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541b05d0 con 0x561829dbce40 2024-05-10T21:18:19.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541af830 con 0x561829f3ac40 2024-05-10T21:18:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc249400700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc2679a5b80 10 client.0 _ll_register_callbacks cb 0x561829e03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:19.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.917+0000 7fc252800700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f5b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 -- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 msgr2=0x561829f4ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f4ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 -- 172.21.15.50:0/4098367807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 msgr2=0x561829f5b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f5b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561829f5b8f0 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc249400700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f4ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc251e00700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc24400f010 tx=0x7fc2440a8160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474d62e835fc9ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc237e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a90f0 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fc244004850 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.918+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2440b1f50 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc244087070 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 10 client.26939 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561829fbe710 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2440b1f50 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc2440dfc70 con 0x561829dbce40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 10 client.26939 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:19.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 10 client.26939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-10T21:18:19.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 20 client.26939 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:19.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.919+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829fbeb80 0x561829fc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:19.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.920+0000 7fc2679a5b80 4 client.26939 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:18:19.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.920+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 --> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561829e6a400 con 0x561829fbeb80 2024-05-10T21:18:19.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.920+0000 7fc249400700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829fbeb80 0x561829fc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.920+0000 7fc249400700 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829fbeb80 0x561829fc0fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b9cad9f2824332d server_cookie=f5ff6cbbd07d70d0 in_seq=0 out_seq=0 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.920+0000 7fc237e00700 10 client.26939 ms_handle_connect on v2:172.21.15.142:6838/3522449943 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc237e00700 1 -- 172.21.15.50:0/4098367807 <== mds.2 v2:172.21.15.142:6838/3522449943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561829e6a400 con 0x561829fbeb80 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc237e00700 10 client.26939 handle_command_reply: tid=0 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 client.26939 shutdown 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 msgr2=0x561829f4c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc24400f010 tx=0x7fc2440a8160 comp rx=0 tx=0).stop 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 shutdown_connections 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:6838/3522449943,v1:172.21.15.142:6839/3522449943] conn(0x561829fbeb80 0x561829fc0fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561829dbce40 0x561829f4c8a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561829f40a80 0x561829f5b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 --2- 172.21.15.50:0/4098367807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561829f3ac40 0x561829f4ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc237e00700 0 client.26939 ms_handle_reset on v2:172.21.15.142:6838/3522449943 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 >> 172.21.15.50:0/4098367807 conn(0x561829e01f70 msgr2=0x561829f2f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 shutdown_connections 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.921+0000 7fc2679a5b80 1 -- 172.21.15.50:0/4098367807 wait complete. 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.922+0000 7fc2679a5b80 20 client.26939 trim_cache size 0 max 16384 2024-05-10T21:18:19.918 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.922+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 msgr2=0x7fc2380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.922+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 0x7fc2380b4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097a0 tx=0x7fc24c009990 comp rx=0 tx=0).stop 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.922+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 msgr2=0x7fc25419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.922+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c370 tx=0x7fc24400c680 comp rx=0 tx=0).stop 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 shutdown_connections 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fc2380b2900 0x7fc2380b4e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc254110a40 0x7fc2541a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc254075ae0 0x7fc25419ce80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 --2- 172.21.15.50:0/2165558664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc254076430 0x7fc25419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:19.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 >> 172.21.15.50:0/2165558664 conn(0x7fc254079c20 msgr2=0x7fc254100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:19.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 shutdown_connections 2024-05-10T21:18:19.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:19.923+0000 7fc253200700 1 -- 172.21.15.50:0/2165558664 wait complete. 2024-05-10T21:18:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.161+0000 7f88c6400640 1 -- 172.21.15.50:0/1950553940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0105cb0 msgr2=0x7f88c01069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.161+0000 7f88c6400640 1 --2- 172.21.15.50:0/1950553940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0105cb0 0x7f88c01069b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b402a3f0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.161+0000 7f88c6400640 1 -- 172.21.15.50:0/1950553940 shutdown_connections 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.161+0000 7f88c6400640 1 --2- 172.21.15.50:0/1950553940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c010bbf0 0x7f88c010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.161+0000 7f88c6400640 1 --2- 172.21.15.50:0/1950553940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c0106ef0 0x7f88c0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.162+0000 7f88c6400640 1 --2- 172.21.15.50:0/1950553940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0105cb0 0x7f88c01069b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.162+0000 7f88c6400640 1 -- 172.21.15.50:0/1950553940 >> 172.21.15.50:0/1950553940 conn(0x7f88c0100ec0 msgr2=0x7f88c0103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.162+0000 7f88c6400640 1 -- 172.21.15.50:0/1950553940 shutdown_connections 2024-05-10T21:18:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.162+0000 7f88c6400640 1 -- 172.21.15.50:0/1950553940 wait complete. 2024-05-10T21:18:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.163+0000 7f88c6400640 1 Processor -- start 2024-05-10T21:18:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.163+0000 7f88c6400640 1 -- start start 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0106ef0 0x7f88c019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c010bbf0 0x7f88c01a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01117b0 con 0x7f88c010bbf0 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0111630 con 0x7f88c0106ef0 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0111930 con 0x7f88c0105cb0 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55192/0 (socket says 172.21.15.50:55192) 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c5000640 1 -- 172.21.15.50:0/4253806405 learned_addr learned my addr 172.21.15.50:0/4253806405 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88bfe00640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0106ef0 0x7f88c019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.164+0000 7f88c5a00640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c010bbf0 0x7f88c01a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 -- 172.21.15.50:0/4253806405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0106ef0 msgr2=0x7f88c019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0106ef0 0x7f88c019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 -- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c010bbf0 msgr2=0x7f88c01a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c010bbf0 0x7f88c01a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 -- 172.21.15.50:0/4253806405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c01a3f30 con 0x7f88c0105cb0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.165+0000 7f88c5000640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f88b4006010 tx=0x7f88b4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.166+0000 7f88bd600640 1 -- 172.21.15.50:0/4253806405 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b403f030 con 0x7f88c0105cb0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.166+0000 7f88bd600640 1 -- 172.21.15.50:0/4253806405 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4031910 con 0x7f88c0105cb0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.166+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c01a41c0 con 0x7f88c0105cb0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.166+0000 7f88bd600640 1 -- 172.21.15.50:0/4253806405 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4035070 con 0x7f88c0105cb0 2024-05-10T21:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.166+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01a4700 con 0x7f88c0105cb0 2024-05-10T21:18:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.170+0000 7f88bd600640 1 -- 172.21.15.50:0/4253806405 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f88b4042030 con 0x7f88c0105cb0 2024-05-10T21:18:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.170+0000 7f88ceb70740 1 Processor -- start 2024-05-10T21:18:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.170+0000 7f88ceb70740 1 -- start start 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d293300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d29a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f7d1f7550 con 0x563f7cef11b0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563f7d19cbc0 con 0x563f7cec82f0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88ceb70740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f7d1f3510 con 0x563f7ced04d0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88c5000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d29a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88c5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d293300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88c5000640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55590/0 (socket says 172.21.15.50:55590) 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88c5000640 1 -- 172.21.15.50:0/948655153 learned_addr learned my addr 172.21.15.50:0/948655153 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88c5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d293300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53676/0 (socket says 172.21.15.50:53676) 2024-05-10T21:18:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.171+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d29a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55208/0 (socket says 172.21.15.50:55208) 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 -- 172.21.15.50:0/948655153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 msgr2=0x563f7d29a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d29a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 -- 172.21.15.50:0/948655153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 msgr2=0x563f7d293300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d293300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 -- 172.21.15.50:0/948655153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7cf09210 con 0x563f7cec82f0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88bd600640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 0x7f88a80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88bd600640 1 -- 172.21.15.50:0/4253806405 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f88b4102450 con 0x7f88c0105cb0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5000640 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88b410a5d0 tx=0x7f88b410a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a38381ad914a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88c5a00640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 0x7f88a80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88b2000640 1 -- 172.21.15.50:0/948655153 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4049070 con 0x563f7cec82f0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88b2000640 1 -- 172.21.15.50:0/948655153 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b40c5270 con 0x563f7cec82f0 2024-05-10T21:18:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.172+0000 7f88b2000640 1 -- 172.21.15.50:0/948655153 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40d2070 con 0x563f7cec82f0 2024-05-10T21:18:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.173+0000 7f88ceb70740 1 -- 172.21.15.50:0/948655153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 msgr2=0x563f7d293880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.173+0000 7f88c5a00640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 0x7f88a80b4d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563f7d2944d0 tx=0x7f88b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.173+0000 7f88ceb70740 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88b410a5d0 tx=0x7f88b410a6c0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 -- 172.21.15.50:0/948655153 shutdown_connections 2024-05-10T21:18:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d29a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cec82f0 0x563f7d293880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 --2- 172.21.15.50:0/948655153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d293300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 -- 172.21.15.50:0/948655153 >> 172.21.15.50:0/948655153 conn(0x563f7cee5300 msgr2=0x563f7d28ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 -- 172.21.15.50:0/948655153 shutdown_connections 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.174+0000 7f88ceb70740 1 -- 172.21.15.50:0/948655153 wait complete. 2024-05-10T21:18:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.175+0000 7f88ceb70740 1 -- start start 2024-05-10T21:18:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.175+0000 7f88ceb70740 1 Processor -- start 2024-05-10T21:18:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7ced04d0 0x563f7d29d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f7d29af90 con 0x563f7ced04d0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563f7d1b0660 con 0x563f7cef11b0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563f7d29b110 con 0x563f7cec82f0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88ceb70740 10 client.0 _ll_register_callbacks cb 0x563f7d17a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55216/0 (socket says 172.21.15.50:55216) 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88c5000640 1 -- 172.21.15.50:0/2149441323 learned_addr learned my addr 172.21.15.50:0/2149441323 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.176+0000 7f88c5a00640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88bfe00640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7ced04d0 0x563f7d29d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 -- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 msgr2=0x563f7d2a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 -- 172.21.15.50:0/2149441323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7ced04d0 msgr2=0x563f7d29d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7ced04d0 0x563f7d29d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 -- 172.21.15.50:0/2149441323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7cfd5620 con 0x563f7cec82f0 2024-05-10T21:18:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88c5000640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f88b410a5a0 tx=0x7f88b40314c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03f0205b0432ea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4049070 con 0x563f7cec82f0 2024-05-10T21:18:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f88b40c5d80 con 0x563f7cec82f0 2024-05-10T21:18:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.177+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b40c5270 con 0x563f7cec82f0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f88b40e3050 con 0x563f7cec82f0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 10 client.26902 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563f7d29af90 con 0x563f7cec82f0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b40c5270 con 0x563f7cec82f0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f88b403ba30 con 0x563f7cec82f0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 10 client.26902 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 20 client.26902 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ceafd80 0x563f7d2aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 4 client.26902 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.178+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f7d205160 con 0x563f7ceafd80 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.179+0000 7f88bfe00640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ceafd80 0x563f7d2aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.179+0000 7f88bfe00640 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ceafd80 0x563f7d2aa490 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb08a8015e8e050 server_cookie=5b25d6cd35dca3ff in_seq=0 out_seq=0 2024-05-10T21:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.179+0000 7f88a6a00640 10 client.26902 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.180+0000 7f88a6a00640 1 -- 172.21.15.50:0/2149441323 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f7d205160 con 0x563f7ceafd80 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.180+0000 7f88a6a00640 10 client.26902 handle_command_reply: tid=0 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.180+0000 7f88ceb70740 1 client.26902 shutdown 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 msgr2=0x563f7d29cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f88b410a5a0 tx=0x7f88b40314c0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 shutdown_connections 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ceafd80 0x563f7d2aa490 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7ced04d0 0x563f7d29d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 --2- 172.21.15.50:0/2149441323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d29cb50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88a6a00640 0 client.26902 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 >> 172.21.15.50:0/2149441323 conn(0x563f7cee5300 msgr2=0x563f7d291ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 shutdown_connections 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 1 -- 172.21.15.50:0/2149441323 wait complete. 2024-05-10T21:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.181+0000 7f88ceb70740 20 client.26902 trim_cache size 0 max 16384 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 Processor -- start 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 -- start start 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cec82f0 0x563f7d29ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01afd30 con 0x563f7cec82f0 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01ad540 con 0x563f7cef11b0 2024-05-10T21:18:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88ceb70740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01ab390 con 0x563f7ced04d0 2024-05-10T21:18:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.206+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:55220/0 (socket says 172.21.15.50:55220) 2024-05-10T21:18:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 -- 172.21.15.50:0/199876924 learned_addr learned my addr 172.21.15.50:0/199876924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88c5000640 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cec82f0 0x563f7d29ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88c5a00640 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 -- 172.21.15.50:0/199876924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 msgr2=0x563f7d2a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 -- 172.21.15.50:0/199876924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cec82f0 msgr2=0x563f7d29ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cec82f0 0x563f7d29ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 -- 172.21.15.50:0/199876924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7d29a640 con 0x563f7ced04d0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.207+0000 7f88bfe00640 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563f7d28fa20 tx=0x7f88ac005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9825be9dcb765b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.208+0000 7f88b1600640 1 -- 172.21.15.50:0/199876924 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac01e500 con 0x563f7ced04d0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.208+0000 7f88b1600640 1 -- 172.21.15.50:0/199876924 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac022020 con 0x563f7ced04d0 2024-05-10T21:18:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.208+0000 7f88b1600640 1 -- 172.21.15.50:0/199876924 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00fcf0 con 0x563f7ced04d0 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- 172.21.15.50:0/199876924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 msgr2=0x563f7d2a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563f7d28fa20 tx=0x7f88ac005f90 comp rx=0 tx=0).stop 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- 172.21.15.50:0/199876924 shutdown_connections 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7ced04d0 0x563f7d2a3f80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7cef11b0 0x563f7d2a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 --2- 172.21.15.50:0/199876924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cec82f0 0x563f7d29ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- 172.21.15.50:0/199876924 >> 172.21.15.50:0/199876924 conn(0x563f7cee5300 msgr2=0x563f7d29c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- 172.21.15.50:0/199876924 shutdown_connections 2024-05-10T21:18:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- 172.21.15.50:0/199876924 wait complete. 2024-05-10T21:18:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.209+0000 7f88ceb70740 1 -- start start 2024-05-10T21:18:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 Processor -- start 2024-05-10T21:18:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d2a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d310830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01aba00 con 0x563f7cef11b0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01ace70 con 0x563f7ced04d0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01ac0d0 con 0x563f7cec82f0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.210+0000 7f88ceb70740 10 client.0 _ll_register_callbacks cb 0x7f88b8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d2a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:55612/0 (socket says 172.21.15.50:55612) 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 -- 172.21.15.50:0/3726964283 learned_addr learned my addr 172.21.15.50:0/3726964283 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88c5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d310830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 -- 172.21.15.50:0/3726964283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 msgr2=0x563f7d2a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d2a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 -- 172.21.15.50:0/3726964283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 msgr2=0x563f7d310830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d310830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88bfe00640 1 -- 172.21.15.50:0/3726964283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7d310da0 con 0x563f7ced04d0 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.211+0000 7f88c5000640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d2a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88bfe00640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f88ac024040 tx=0x7f88ac0114f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ebdcac1e1d2833 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02b070 con 0x563f7ced04d0 2024-05-10T21:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f88ac00fc70 con 0x563f7ced04d0 2024-05-10T21:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac022020 con 0x563f7ced04d0 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f88ac035050 con 0x563f7ced04d0 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88ceb70740 10 client.26957 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.212+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563f7d312000 con 0x563f7ced04d0 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac022020 con 0x563f7ced04d0 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f88ac0054a0 con 0x563f7ced04d0 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 10 client.26957 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 20 client.26957 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ce851c0 0x563f7d314a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 4 client.26957 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f7d1fefd0 con 0x563f7ce851c0 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.213+0000 7f88c5000640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ce851c0 0x563f7d314a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.214+0000 7f88c5000640 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ce851c0 0x563f7d314a80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd496d41ca3d03c server_cookie=20c6b26726baebc in_seq=0 out_seq=0 2024-05-10T21:18:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.214+0000 7f88a6a00640 10 client.26957 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88a6a00640 1 -- 172.21.15.50:0/3726964283 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f7d1fefd0 con 0x563f7ce851c0 2024-05-10T21:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88a6a00640 10 client.26957 handle_command_reply: tid=0 2024-05-10T21:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 client.26957 shutdown 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 msgr2=0x563f7d2a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f88ac024040 tx=0x7f88ac0114f0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 shutdown_connections 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x563f7ce851c0 0x563f7d314a80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f7cef11b0 0x563f7d310830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88a6a00640 0 client.26957 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f7ced04d0 0x563f7d2a1d20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 --2- 172.21.15.50:0/3726964283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563f7cec82f0 0x563f7d2a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 >> 172.21.15.50:0/3726964283 conn(0x563f7cee5300 msgr2=0x563f7d291a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.220+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 shutdown_connections 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.221+0000 7f88ceb70740 1 -- 172.21.15.50:0/3726964283 wait complete. 2024-05-10T21:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.221+0000 7f88ceb70740 20 client.26957 trim_cache size 0 max 16384 2024-05-10T21:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 msgr2=0x7f88a80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 0x7f88a80b4d70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563f7d2944d0 tx=0x7f88b8009500 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 msgr2=0x7f88c019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f88b4006010 tx=0x7f88b4004270 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 shutdown_connections 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f88a80b2860 0x7f88a80b4d70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88c010bbf0 0x7f88c01a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c0106ef0 0x7f88c019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 --2- 172.21.15.50:0/4253806405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88c0105cb0 0x7f88c019c3f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 >> 172.21.15.50:0/4253806405 conn(0x7f88c0100ec0 msgr2=0x7f88c0107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 shutdown_connections 2024-05-10T21:18:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:20.222+0000 7f88c6400640 1 -- 172.21.15.50:0/4253806405 wait complete. 2024-05-10T21:18:20.360 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-05-10T21:18:20.360 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T21:18:20.971 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T21:18:21.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:21 smithi050 ceph-mon[29546]: pgmap v746: 97 pgs: 97 active+clean; 264 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 660 KiB/s wr, 265 op/s 2024-05-10T21:18:21.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:21 smithi092 ceph-mon[34415]: pgmap v746: 97 pgs: 97 active+clean; 264 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 660 KiB/s wr, 265 op/s 2024-05-10T21:18:21.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:21 smithi142 ceph-mon[34846]: pgmap v746: 97 pgs: 97 active+clean; 264 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 660 KiB/s wr, 265 op/s 2024-05-10T21:18:21.435 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:21.604 DEBUG:teuthology.parallel:result is None 2024-05-10T21:18:21.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T21:18:21.685 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T21:18:21.685 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-10T21:18:21.697 INFO:tasks.quiescer:joining Quiescers 2024-05-10T21:18:21.697 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-10T21:18:21.697 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:18:22.967 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:23.150 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:23 smithi050 ceph-mon[29546]: pgmap v747: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-05-10T21:18:23.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:23 smithi092 ceph-mon[34415]: pgmap v747: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-05-10T21:18:23.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:23 smithi142 ceph-mon[34846]: pgmap v747: 97 pgs: 97 active+clean; 255 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 -- 172.21.15.50:0/2564699458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 msgr2=0x7fef1c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 --2- 172.21.15.50:0/2564699458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c108200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef04009990 tx=0x7fef04009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 -- 172.21.15.50:0/2564699458 shutdown_connections 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 --2- 172.21.15.50:0/2564699458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 0x7fef1c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 --2- 172.21.15.50:0/2564699458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.419+0000 7fef21200700 1 --2- 172.21.15.50:0/2564699458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c108200 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.420+0000 7fef21200700 1 -- 172.21.15.50:0/2564699458 >> 172.21.15.50:0/2564699458 conn(0x7fef1c0fedc0 msgr2=0x7fef1c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.420+0000 7fef21200700 1 -- 172.21.15.50:0/2564699458 shutdown_connections 2024-05-10T21:18:24.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.420+0000 7fef21200700 1 -- 172.21.15.50:0/2564699458 wait complete. 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.421+0000 7fef21200700 1 Processor -- start 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.421+0000 7fef21200700 1 -- start start 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.421+0000 7fef21200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.421+0000 7fef21200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef21200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 0x7fef1c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef21200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c113350 con 0x7fef1c1034c0 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef21200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1131d0 con 0x7fef1c107df0 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef21200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1134d0 con 0x7fef1c108740 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59424/0 (socket says 172.21.15.50:59424) 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19e00700 1 -- 172.21.15.50:0/2160969500 learned_addr learned my addr 172.21.15.50:0/2160969500 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:24.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19400700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef1a800700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 0x7fef1c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19400700 1 -- 172.21.15.50:0/2160969500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 msgr2=0x7fef1c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19400700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 0x7fef1c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19400700 1 -- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 msgr2=0x7fef1c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.422+0000 7fef19400700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef19400700 1 -- 172.21.15.50:0/2160969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c1a1eb0 con 0x7fef1c107df0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef19e00700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef19400700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c19d580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04000c00 tx=0x7fef04005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef12a00700 1 -- 172.21.15.50:0/2160969500 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04026070 con 0x7fef1c107df0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef12a00700 1 -- 172.21.15.50:0/2160969500 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0400fd70 con 0x7fef1c107df0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef12a00700 1 -- 172.21.15.50:0/2160969500 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040049f0 con 0x7fef1c107df0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c1a2140 con 0x7fef1c107df0 2024-05-10T21:18:24.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.423+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c1a9ac0 con 0x7fef1c107df0 2024-05-10T21:18:24.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.427+0000 7fef12a00700 1 -- 172.21.15.50:0/2160969500 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef04029070 con 0x7fef1c107df0 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.429+0000 7fef12a00700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 0x7fef000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.429+0000 7fef2fd42b80 1 Processor -- start 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.429+0000 7fef12a00700 1 -- 172.21.15.50:0/2160969500 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fef040e2d00 con 0x7fef1c107df0 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.429+0000 7fef2fd42b80 1 -- start start 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.429+0000 7fef19e00700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 0x7fef000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee26e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602ded48a20 con 0x5602dee269f0 2024-05-10T21:18:24.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5602dece6760 con 0x5602dee2c830 2024-05-10T21:18:24.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602ded449e0 con 0x5602deca8e40 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef19400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef19400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50816/0 (socket says 172.21.15.50:50816) 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef19e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef19400700 1 -- 172.21.15.50:0/1340365420 learned_addr learned my addr 172.21.15.50:0/1340365420 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef1a800700 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee26e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.430+0000 7fef19e00700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 0x7fef000b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0104e0 tx=0x7fef0c007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 -- 172.21.15.50:0/1340365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 msgr2=0x5602dee2c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 -- 172.21.15.50:0/1340365420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 msgr2=0x5602dee26e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee26e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 -- 172.21.15.50:0/1340365420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602dec958d0 con 0x5602dee2c830 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19e00700 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef19400700 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef040a6400 tx=0x7fef040a9f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bfc9c477b2a8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef0b400700 1 -- 172.21.15.50:0/1340365420 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040ae730 con 0x5602dee2c830 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef0b400700 1 -- 172.21.15.50:0/1340365420 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef040b3150 con 0x5602dee2c830 2024-05-10T21:18:24.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.431+0000 7fef0b400700 1 -- 172.21.15.50:0/1340365420 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040a9840 con 0x5602dee2c830 2024-05-10T21:18:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1340365420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 msgr2=0x5602dee264b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef040a6400 tx=0x7fef040a9f60 comp rx=0 tx=0).stop 2024-05-10T21:18:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1340365420 shutdown_connections 2024-05-10T21:18:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee26e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1340365420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee264b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.432+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1340365420 >> 172.21.15.50:0/1340365420 conn(0x5602decedf70 msgr2=0x5602dee212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.433+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1340365420 shutdown_connections 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.433+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1340365420 wait complete. 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.433+0000 7fef2fd42b80 1 -- start start 2024-05-10T21:18:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.433+0000 7fef2fd42b80 1 Processor -- start 2024-05-10T21:18:24.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602dee330d0 con 0x5602dee269f0 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5602decaeb20 con 0x5602dee2c830 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602dee33250 con 0x5602deca8e40 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.434+0000 7fef2fd42b80 10 client.0 _ll_register_callbacks cb 0x5602decefc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef19400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50832/0 (socket says 172.21.15.50:50832) 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 -- 172.21.15.50:0/1035361601 learned_addr learned my addr 172.21.15.50:0/1035361601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef19e00700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 -- 172.21.15.50:0/1035361601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 msgr2=0x5602dee2ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 -- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 msgr2=0x5602dee304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 -- 172.21.15.50:0/1035361601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ded0e420 con 0x5602dee2c830 2024-05-10T21:18:24.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7fef1a800700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef14011f60 tx=0x7fef14014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26ca90d6dee2ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.435+0000 7feeffe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:24.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14014ba0 con 0x5602dee2c830 2024-05-10T21:18:24.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fef14014d40 con 0x5602dee2c830 2024-05-10T21:18:24.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef040b3150 con 0x5602dee2c830 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fef14025070 con 0x5602dee2c830 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7fef2fd42b80 10 client.26969 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5602dee3dbb0 con 0x5602dee2c830 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.436+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef040b3150 con 0x5602dee2c830 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fef14007500 con 0x5602dee2c830 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 10 client.26969 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 10 client.26969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:18:24.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 20 client.26969 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602dee40240 0x5602dee42660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 4 client.26969 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602dedf31c0 con 0x5602dee40240 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.437+0000 7fef19e00700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602dee40240 0x5602dee42660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.438+0000 7fef19e00700 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602dee40240 0x5602dee42660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=951447c378982d79 server_cookie=e194ff2273a3df5a in_seq=0 out_seq=0 2024-05-10T21:18:24.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.438+0000 7feeffe00700 10 client.26969 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7feeffe00700 1 -- 172.21.15.50:0/1035361601 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602dedf31c0 con 0x5602dee40240 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7feeffe00700 10 client.26969 handle_command_reply: tid=0 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 client.26969 shutdown 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 msgr2=0x5602dee30a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef14011f60 tx=0x7fef14014600 comp rx=0 tx=0).stop 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 shutdown_connections 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee30a00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602dee40240 0x5602dee42660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee2ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1035361601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 >> 172.21.15.50:0/1035361601 conn(0x5602decedf70 msgr2=0x5602dee26e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7feeffe00700 0 client.26969 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 shutdown_connections 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.439+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1035361601 wait complete. 2024-05-10T21:18:24.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.440+0000 7fef2fd42b80 20 client.26969 trim_cache size 0 max 16384 2024-05-10T21:18:24.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 Processor -- start 2024-05-10T21:18:24.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 -- start start 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee3c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b3500 con 0x5602dee269f0 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b0cd0 con 0x5602dee2c830 2024-05-10T21:18:24.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1aeb80 con 0x5602deca8e40 2024-05-10T21:18:24.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.457+0000 7fef19400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee3c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee3c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59450/0 (socket says 172.21.15.50:59450) 2024-05-10T21:18:24.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19400700 1 -- 172.21.15.50:0/1193823132 learned_addr learned my addr 172.21.15.50:0/1193823132 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:24.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef1a800700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 -- 172.21.15.50:0/1193823132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 msgr2=0x5602dee304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 -- 172.21.15.50:0/1193823132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 msgr2=0x5602dee3c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee3c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 -- 172.21.15.50:0/1193823132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602dee2c640 con 0x5602deca8e40 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef1a800700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.458+0000 7fef19e00700 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee264b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5602dee42d60 tx=0x7fef0c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94dc989369d9f960 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.459+0000 7fef0aa00700 1 -- 172.21.15.50:0/1193823132 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c030070 con 0x5602deca8e40 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.459+0000 7fef0aa00700 1 -- 172.21.15.50:0/1193823132 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0c02a8e0 con 0x5602deca8e40 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.459+0000 7fef0aa00700 1 -- 172.21.15.50:0/1193823132 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c01e420 con 0x5602deca8e40 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1193823132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 msgr2=0x5602dee264b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee264b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5602dee42d60 tx=0x7fef0c013000 comp rx=0 tx=0).stop 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1193823132 shutdown_connections 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602dee2c830 0x5602dee304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602deca8e40 0x5602dee264b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/1193823132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1193823132 >> 172.21.15.50:0/1193823132 conn(0x5602decedf70 msgr2=0x5602dee212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1193823132 shutdown_connections 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- 172.21.15.50:0/1193823132 wait complete. 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.460+0000 7fef2fd42b80 1 -- start start 2024-05-10T21:18:24.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 Processor -- start 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 0x5602dee47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1af1d0 con 0x5602dee269f0 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b0600 con 0x5602deca8e40 2024-05-10T21:18:24.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1af860 con 0x5602dee2c830 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef2fd42b80 10 client.0 _ll_register_callbacks cb 0x5602decefc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef19e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.461+0000 7fef19400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50848/0 (socket says 172.21.15.50:50848) 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 -- 172.21.15.50:0/3265715046 learned_addr learned my addr 172.21.15.50:0/3265715046 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59466/0 (socket says 172.21.15.50:59466) 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef1a800700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 0x5602dee47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 -- 172.21.15.50:0/3265715046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 msgr2=0x5602dee47770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 0x5602dee47770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 -- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 msgr2=0x5602dee47230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:24.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 -- 172.21.15.50:0/3265715046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602dee47d30 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19400700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef1a800700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 0x5602dee47770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7fef19e00700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef0c010c30 tx=0x7fef0c0201d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6a06198bc88626 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.462+0000 7feeffe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c030040 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fef0c02a630 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0c02a8e0 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fef0c005040 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7fef2fd42b80 10 client.26975 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.463+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5602deeaa700 con 0x5602deca8e40 2024-05-10T21:18:24.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0c02a8e0 con 0x5602deca8e40 2024-05-10T21:18:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fef0c0046f0 con 0x5602deca8e40 2024-05-10T21:18:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 10 client.26975 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 10 client.26975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-10T21:18:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 20 client.26975 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602deeaad40 0x5602deead160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 4 client.26975 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.464+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 --> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602ded56400 con 0x5602deeaad40 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.465+0000 7fef19400700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602deeaad40 0x5602deead160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.465+0000 7fef19400700 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602deeaad40 0x5602deead160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae034187f77c48ff server_cookie=aea179d49c07b53e in_seq=0 out_seq=0 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.465+0000 7feeffe00700 10 client.26975 ms_handle_connect on v2:172.21.15.50:6840/908891204 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.468+0000 7feeffe00700 1 -- 172.21.15.50:0/3265715046 <== mds.1 v2:172.21.15.50:6840/908891204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+119114 (crc 0 0 0) 0x5602ded56400 con 0x5602deeaad40 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.468+0000 7feeffe00700 10 client.26975 handle_command_reply: tid=0 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 client.26975 shutdown 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 msgr2=0x5602dee38820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef0c010c30 tx=0x7fef0c0201d0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 shutdown_connections 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602deca8e40 0x5602dee38820 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:6840/908891204,v1:172.21.15.50:6841/908891204] conn(0x5602deeaad40 0x5602deead160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602dee2c830 0x5602dee47770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 --2- 172.21.15.50:0/3265715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602dee269f0 0x5602dee47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7feeffe00700 0 client.26975 ms_handle_reset on v2:172.21.15.50:6840/908891204 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 >> 172.21.15.50:0/3265715046 conn(0x5602decedf70 msgr2=0x5602dee212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 shutdown_connections 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 1 -- 172.21.15.50:0/3265715046 wait complete. 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.469+0000 7fef2fd42b80 20 client.26975 trim_cache size 0 max 16384 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.469 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.470 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 27251176, 2024-05-10T21:18:24.470 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266368, 2024-05-10T21:18:24.470 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg" 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 41781722, 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264776, 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml" 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 51077255, 2024-05-10T21:18:24.471 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264600, 2024-05-10T21:18:24.472 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg" 2024-05-10T21:18:24.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.472 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.472 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 68773459, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264171, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml" 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 70453880, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264816, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml" 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 79329381, 2024-05-10T21:18:24.473 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266100, 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml" 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 104176075, 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264826, 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml" 2024-05-10T21:18:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 108379976, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631119, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml" 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 117266632, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631033, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml" 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.475 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 122773241, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264149, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml" 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 125915668, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511633027, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml" 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 132772991, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630999, 2024-05-10T21:18:24.476 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml" 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 133488554, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264932, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg" 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 134284715, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264148, 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml" 2024-05-10T21:18:24.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 144446297, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266127, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg" 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 152945887, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630992, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml" 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.478 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 153471331, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264814, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml" 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 153767616, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631141, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml" 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 166668602, 2024-05-10T21:18:24.479 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264871, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg" 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 172343981, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266025, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml" 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 172899425, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266130, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg" 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 182392436, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632166, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml" 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 200445241, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631121, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml" 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.481 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 206968987, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023270879, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml" 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 219471537, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266308, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg" 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.482 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 221035555, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631034, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml" 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 221643951, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631144, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml" 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 226673575, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266281, 2024-05-10T21:18:24.483 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg" 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 232040564, 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265960, 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml" 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.484 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 234127358, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264562, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg" 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 236578561, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266189, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg" 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 249690468, 2024-05-10T21:18:24.485 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266084, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml" 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 251265029, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265951, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml" 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 259342464, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264529, 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg" 2024-05-10T21:18:24.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 267529433, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630968, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml" 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 296462030, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631016, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml" 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.487 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 299179713, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264272, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml" 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 301577335, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265943, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml" 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 305158190, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266076, 2024-05-10T21:18:24.488 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml" 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 321310393, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631237, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg" 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 325684567, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264660, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg" 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.489 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 345492563, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264213, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml" 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 352322637, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264960, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg" 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 352599097, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264967, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml" 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 363645441, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264202, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml" 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 365266567, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264271, 2024-05-10T21:18:24.491 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml" 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 382467249, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266068, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml" 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 394714925, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264731, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml" 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.492 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 400868388, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630970, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml" 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 402650920, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266290, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg" 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 403741959, 2024-05-10T21:18:24.493 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266115, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg" 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 405245706, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632169, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml" 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 424140301, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630998, 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml" 2024-05-10T21:18:24.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 442998730, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631151, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml" 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 458641551, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264802, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml" 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.495 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 471303893, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630982, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml" 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 485265787, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631103, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml" 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 498219331, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264790, 2024-05-10T21:18:24.496 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml" 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 501364449, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631283, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml" 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 516669425, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264556, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg" 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.497 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 525251196, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264690, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg" 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 529865820, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264229, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml" 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.498 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 531395852, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631148, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml" 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 538682343, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266329, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg" 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 547333965, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631160, 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml" 2024-05-10T21:18:24.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 564974996, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266347, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg" 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 575850797, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264719, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml" 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.501 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 578784361, 2024-05-10T21:18:24.501 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266181, 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg" 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 581979204, 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632162, 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml" 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.502 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 601493855, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630973, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml" 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 626827740, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630985, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml" 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.503 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 630201590, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630974, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml" 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 632790521, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631017, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml" 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.504 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 633335506, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264127, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml" 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 635807912, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266326, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg" 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 642442537, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264168, 2024-05-10T21:18:24.505 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml" 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 642983058, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264503, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg" 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 666032959, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266064, 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml" 2024-05-10T21:18:24.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 672449157, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264788, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml" 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 684607469, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264350, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml" 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 691919846, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264579, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg" 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 700421982, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632163, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml" 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 730195983, 2024-05-10T21:18:24.508 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631125, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml" 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 739211789, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264247, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml" 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 740840225, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630963, 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml" 2024-05-10T21:18:24.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 743339957, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630964, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml" 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 745027693, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266350, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg" 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.510 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 751677761, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266039, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml" 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 768844333, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264210, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml" 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 782638177, 2024-05-10T21:18:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631219, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml" 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 801404940, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631013, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml" 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 815482504, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266060, 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml" 2024-05-10T21:18:24.512 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 816911223, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264796, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml" 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 822557670, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264333, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml" 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.513 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 822687363, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266213, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg" 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 846939547, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264391, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml" 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 848703443, 2024-05-10T21:18:24.514 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630995, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml" 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 849786807, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264208, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml" 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 852223549, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631120, 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml" 2024-05-10T21:18:24.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 852745254, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631241, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml" 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 863342259, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266145, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg" 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.516 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 902622058, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264207, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml" 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 904920638, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264404, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg" 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 916113510, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264830, 2024-05-10T21:18:24.517 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml" 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 916177110, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264681, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg" 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 923841343, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630996, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml" 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.518 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 926953038, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631154, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml" 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 929868263, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265921, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml" 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.519 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 941980520, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631111, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml" 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 952233161, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630971, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml" 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 967572611, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264416, 2024-05-10T21:18:24.520 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg" 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 968018449, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266133, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg" 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 968417754, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264592, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg" 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.521 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 974071158, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631145, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml" 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 989323725, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265956, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml" 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 996569383, 2024-05-10T21:18:24.522 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265987, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml" 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 998482170, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631236, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml" 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1022287994, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266178, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg" 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.523 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1025968701, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265918, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml" 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1026303607, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265932, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml" 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1033869196, 2024-05-10T21:18:24.524 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266032, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml" 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1036567735, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264810, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml" 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1038771223, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264804, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml" 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1051386037, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265962, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml" 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1054995731, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264294, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml" 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.526 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1071144628, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630993, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml" 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1084531862, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265947, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml" 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1109560301, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631164, 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml" 2024-05-10T21:18:24.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1116059498, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265996, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml" 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1134077563, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631108, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml" 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.528 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1142650262, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630984, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml" 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1145967154, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631029, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml" 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1164255611, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266118, 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg" 2024-05-10T21:18:24.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1164801078, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266275, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg" 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1167818205, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631132, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml" 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.530 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1170625093, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630981, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml" 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1174411399, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266028, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml" 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1176058050, 2024-05-10T21:18:24.531 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264764, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml" 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1183368398, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630961, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml" 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1184045967, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264532, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg" 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.532 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1190749089, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631014, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml" 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1191135154, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264853, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg" 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.533 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1199226691, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266106, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml" 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1201884609, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266022, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml" 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1202467734, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265972, 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml" 2024-05-10T21:18:24.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1241967127, 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630977, 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml" 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1257462865, 2024-05-10T21:18:24.535 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266121, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg" 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1277205586, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631146, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml" 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1319573693, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264792, 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml" 2024-05-10T21:18:24.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1344195742, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631225, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml" 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1365306278, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631161, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml" 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.537 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1374324174, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264142, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml" 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1395129948, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264422, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg" 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1414637287, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264824, 2024-05-10T21:18:24.538 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml" 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1435093493, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264768, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml" 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1439100688, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266243, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg" 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.539 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1444249096, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631008, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml" 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1459409690, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266148, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg" 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.540 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1462154511, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511633699, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml" 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1469172104, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631159, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml" 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.541 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1478589130, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266253, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg" 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1482537863, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266284, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg" 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1487841752, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632164, 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml" 2024-05-10T21:18:24.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1493625125, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631010, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml" 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1493937181, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631015, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml" 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.543 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1501777228, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264909, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg" 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1509349008, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631165, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml" 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1521927933, 2024-05-10T21:18:24.544 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264468, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg" 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1522125727, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266072, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml" 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1531103205, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264373, 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml" 2024-05-10T21:18:24.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1568637904, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264137, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml" 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1570919360, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631131, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml" 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1570988358, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266018, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml" 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1596237404, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266092, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml" 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1604481995, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631026, 2024-05-10T21:18:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml" 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1612709126, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264778, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml" 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1624577820, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264917, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg" 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1637187462, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264749, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml" 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1642192697, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631135, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml" 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1644841509, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264822, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml" 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1653257842, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631134, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml" 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1676945840, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266311, 2024-05-10T21:18:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg" 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1690739389, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264842, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml" 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1696412947, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631110, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml" 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1706768814, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265910, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml" 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1708166924, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264187, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml" 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1709397460, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631037, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml" 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1723340432, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266359, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg" 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1729209212, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266207, 2024-05-10T21:18:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg" 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1742366263, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264291, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml" 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1746145650, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630972, 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml" 2024-05-10T21:18:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1746435824, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630991, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml" 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1747599049, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264786, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml" 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1755123310, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631102, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml" 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1755657843, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266380, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg" 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1763656622, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632418, 2024-05-10T21:18:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml" 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1766084662, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264654, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg" 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1769186122, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266172, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg" 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1772521493, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264146, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml" 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1774188245, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630988, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml" 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1786265401, 2024-05-10T21:18:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264725, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml" 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1794895395, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264747, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml" 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1806853587, 2024-05-10T21:18:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266046, 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml" 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1809399859, 2024-05-10T21:18:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264286, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml" 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1814809798, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631269, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml" 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1826472703, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631248, 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml" 2024-05-10T21:18:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1840699100, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266109, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml" 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1855406871, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264836, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml" 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1859033508, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631022, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml" 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1862542682, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632212, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml" 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1866963239, 2024-05-10T21:18:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630976, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml" 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1885887634, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265984, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml" 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1915409745, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266202, 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg" 2024-05-10T21:18:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1915918834, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264818, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml" 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1919108591, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264832, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml" 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1922759373, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266195, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg" 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1927064360, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631117, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml" 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1929647193, 2024-05-10T21:18:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631133, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml" 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1935881129, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264516, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg" 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1943545615, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631038, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml" 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1952759175, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264885, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg" 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 1995624755, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631139, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml" 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2021739471, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264834, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml" 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2021903939, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266374, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg" 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2028221461, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266383, 2024-05-10T21:18:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg" 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2034531360, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631041, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml" 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2035037605, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264752, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml" 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2035838983, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631112, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml" 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2041598994, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264711, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml" 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2042850268, 2024-05-10T21:18:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266112, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml" 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2050908664, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265936, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml" 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2055731661, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631279, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml" 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2069355589, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631275, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml" 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2069402495, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631030, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml" 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2081696391, 2024-05-10T21:18:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266238, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg" 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2084421152, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631032, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml" 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2085027275, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631113, 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml" 2024-05-10T21:18:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2085186883, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631130, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml" 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2097232361, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264517, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg" 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2098008793, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631012, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml" 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2098410436, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265979, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml" 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2115206893, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632170, 2024-05-10T21:18:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml" 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2118555814, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266353, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg" 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2121873780, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264862, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg" 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2124483121, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266166, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg" 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2124906236, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264755, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml" 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2141597947, 2024-05-10T21:18:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630997, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml" 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2147365898, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266287, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg" 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2148916616, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264840, 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml" 2024-05-10T21:18:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2160741414, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266163, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg" 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2166586825, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264279, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml" 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2174233877, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264875, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg" 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2191319905, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631246, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml" 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2199985986, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631027, 2024-05-10T21:18:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml" 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2205803713, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264676, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg" 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2210245074, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264672, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg" 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2227623846, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266268, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg" 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2238048482, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264772, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml" 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2241330335, 2024-05-10T21:18:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264650, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg" 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2250361952, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264812, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml" 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2250795648, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266139, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg" 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2260664826, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264571, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg" 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2266058140, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266154, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg" 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2272742220, 2024-05-10T21:18:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264363, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml" 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2276406971, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266320, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg" 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2276868298, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264594, 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg" 2024-05-10T21:18:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2285379462, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265022, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml" 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2288802331, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266296, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg" 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2289618155, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264715, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml" 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2294313781, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264320, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml" 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2294957632, 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 msgr2=0x7fef000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 0x7fef000b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0104e0 tx=0x7fef0c007480 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 msgr2=0x7fef1c19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c19d580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04000c00 tx=0x7fef04005950 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 shutdown_connections 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fef000b2900 0x7fef000b4e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1c108740 0x7fef1c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef1c1034c0 0x7fef1c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.506+0000 7fef21200700 1 --2- 172.21.15.50:0/2160969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef1c107df0 0x7fef1c19d580 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.507+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 >> 172.21.15.50:0/2160969500 conn(0x7fef1c0fedc0 msgr2=0x7fef1c10ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:24.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.507+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 shutdown_connections 2024-05-10T21:18:24.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:24.507+0000 7fef21200700 1 -- 172.21.15.50:0/2160969500 wait complete. 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023271811, 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml" 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2297751321, 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264582, 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg" 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2309170005, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264115, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml" 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2313587971, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264547, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg" 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2313879867, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630994, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml" 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2314207020, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631147, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml" 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2328040986, 2024-05-10T21:18:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631001, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml" 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2345564899, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264386, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml" 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2356253608, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631162, 2024-05-10T21:18:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml" 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2379184693, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264926, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg" 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2394025997, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266160, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg" 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2398140620, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023267598, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml" 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2416015364, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631224, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg" 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2435674300, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631000, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml" 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2439515533, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631028, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml" 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2448093355, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264285, 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml" 2024-05-10T21:18:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2451237315, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264806, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml" 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2455648714, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630983, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml" 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2475968859, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631035, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml" 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2480891551, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631021, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml" 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2489373942, 2024-05-10T21:18:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264159, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml" 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2496619841, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266157, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg" 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2501582649, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631115, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml" 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2506799983, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264176, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml" 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2528965882, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631009, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml" 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2544130922, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265975, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml" 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2550039224, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630959, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml" 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2556142876, 2024-05-10T21:18:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266362, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg" 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2563863309, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265998, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml" 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2565055024, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266323, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg" 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2572855861, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023270764, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml" 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2576504882, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631163, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml" 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2582752363, 2024-05-10T21:18:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264574, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg" 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2582804314, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266056, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml" 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2583701166, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266088, 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml" 2024-05-10T21:18:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2603050229, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631007, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml" 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2605383714, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631042, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml" 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2631900879, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630990, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml" 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2632458380, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264828, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml" 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2637594234, 2024-05-10T21:18:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631107, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml" 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2645280923, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264939, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml" 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2653538245, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631114, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml" 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2664564743, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264699, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml" 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2685183466, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264357, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml" 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2687518681, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631004, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml" 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2692964840, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264904, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg" 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2712845126, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631128, 2024-05-10T21:18:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml" 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2714970370, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264800, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml" 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2717761821, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631123, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml" 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2718401193, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266341, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg" 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2720394854, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264330, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml" 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2725946923, 2024-05-10T21:18:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266299, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg" 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2730895197, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264794, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml" 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2735307232, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631171, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml" 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2743610447, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264739, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml" 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2756416505, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511633549, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml" 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2757210753, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266317, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg" 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2760393880, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264412, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg" 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2773348980, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266272, 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg" 2024-05-10T21:18:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2784579931, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630965, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml" 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2793079428, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264782, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml" 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2795872165, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631149, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml" 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2795887759, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632165, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml" 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2821749331, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632414, 2024-05-10T21:18:24.617 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg" 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2824780603, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630960, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml" 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2839467973, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630986, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml" 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.618 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2848642525, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632161, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml" 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2850464593, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264378, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml" 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2873746459, 2024-05-10T21:18:24.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264849, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml" 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2898236511, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264858, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg" 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2913759311, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264844, 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml" 2024-05-10T21:18:24.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2922769514, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264953, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg" 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2927578888, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266233, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg" 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.621 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2928184680, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631003, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml" 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2929969976, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264963, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg" 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2937727187, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631109, 2024-05-10T21:18:24.622 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml" 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2957489322, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631018, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml" 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2959725509, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264400, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg" 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.623 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2974828384, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266142, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg" 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2980347315, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264455, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg" 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2982523891, 2024-05-10T21:18:24.624 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266258, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg" 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2983065204, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630987, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml" 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2988291099, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264770, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml" 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.625 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 2993258638, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266263, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg" 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3001382202, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631124, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml" 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3002667600, 2024-05-10T21:18:24.626 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264880, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg" 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3002864321, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264865, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg" 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3006357152, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631002, 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml" 2024-05-10T21:18:24.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3007121177, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631122, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml" 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3012342749, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266053, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml" 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.628 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3035750679, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264472, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg" 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3061897753, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630958, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml" 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3065256453, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264705, 2024-05-10T21:18:24.629 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml" 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3075262343, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511633698, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg" 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3090534988, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 3298534884586, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml" 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3092647506, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265991, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml" 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3093479070, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631106, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml" 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3097682004, 2024-05-10T21:18:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265982, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml" 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3101157198, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266124, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg" 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3118566112, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264766, 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml" 2024-05-10T21:18:24.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3118923750, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631245, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg" 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3129777249, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264798, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml" 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.633 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3130884427, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631118, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml" 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3131349752, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631101, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml" 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3135791406, 2024-05-10T21:18:24.634 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266293, 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg" 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3148966126, 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631040, 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml" 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.635 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3186600691, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265017, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg" 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3202621922, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632168, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml" 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3215296928, 2024-05-10T21:18:24.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264145, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml" 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3225159793, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264180, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml" 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3226718057, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266305, 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg" 2024-05-10T21:18:24.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3231285753, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630979, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml" 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3232880188, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264308, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml" 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.638 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3241076895, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266371, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg" 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3251905756, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264741, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml" 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3254050727, 2024-05-10T21:18:24.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264694, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml" 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3267675601, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265940, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml" 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3268758051, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264675, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg" 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3298448998, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266042, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml" 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3316549385, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265992, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml" 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.641 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3324145931, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266314, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg" 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3330548757, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632160, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml" 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.642 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3337125182, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266218, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg" 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3337268944, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266228, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg" 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3390164945, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264838, 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml" 2024-05-10T21:18:24.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3404475667, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264577, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg" 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3406984871, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631104, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml" 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.644 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3410264128, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023271015, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml" 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3410789920, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264589, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg" 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3416649720, 2024-05-10T21:18:24.645 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631136, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml" 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3420570030, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631058, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml" 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3431890584, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264925, 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg" 2024-05-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3432732370, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266004, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml" 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3451240814, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264868, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg" 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.647 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3467741241, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631011, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml" 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3482873968, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631025, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml" 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3497056240, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630980, 2024-05-10T21:18:24.648 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml" 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3503388525, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265914, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml" 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3508030218, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266169, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg" 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.649 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3511176430, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266175, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg" 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3513023183, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264627, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg" 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.650 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3513725190, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264234, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml" 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3519371974, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266049, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml" 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3546102657, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023270337, 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml" 2024-05-10T21:18:24.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3559124357, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264942, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg" 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3572412572, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264205, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml" 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.652 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3577050448, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266377, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg" 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3579892750, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264183, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml" 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3583933007, 2024-05-10T21:18:24.653 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266096, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml" 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3588202795, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631036, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml" 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3589647605, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265969, 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml" 2024-05-10T21:18:24.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3592400505, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264855, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg" 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3601339329, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264820, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml" 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.655 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3622220189, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631129, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml" 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3623075762, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264784, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml" 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3651326218, 2024-05-10T21:18:24.656 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266223, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg" 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3657973747, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264293, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml" 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3661938753, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266008, 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml" 2024-05-10T21:18:24.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3674528723, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266151, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg" 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3693790575, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631152, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml" 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.658 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3699126502, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631153, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml" 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3705506085, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266335, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg" 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3711573514, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264371, 2024-05-10T21:18:24.659 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg" 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3730225899, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631039, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml" 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3755756481, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631226, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml" 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.660 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3756118653, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264482, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg" 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3765498747, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264762, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml" 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3773086919, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264313, 2024-05-10T21:18:24.661 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml" 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3778696910, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266278, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg" 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3779616914, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630967, 2024-05-10T21:18:24.662 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml" 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3781765392, 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265929, 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml" 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.666 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3785970329, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630978, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml" 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3787462651, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631105, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml" 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3790727952, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266015, 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml" 2024-05-10T21:18:24.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3795829643, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265925, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml" 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3796639447, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631238, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml" 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.668 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3810723426, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266356, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg" 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3818700748, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631126, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml" 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3828936980, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266103, 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml" 2024-05-10T21:18:24.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3866912066, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630989, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml" 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3889732721, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630966, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml" 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.670 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3893005618, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264709, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml" 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3898045736, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264744, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml" 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3903087160, 2024-05-10T21:18:24.671 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631006, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml" 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3909901693, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631005, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml" 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3921199254, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264257, 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml" 2024-05-10T21:18:24.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3940402659, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264619, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg" 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3944637692, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264120, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml" 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3964301865, 2024-05-10T21:18:24.673 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266035, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml" 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3976098486, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630969, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml" 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3978644520, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264774, 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml" 2024-05-10T21:18:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3989786101, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511632167, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml" 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 3996475894, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631150, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml" 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.675 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4011268582, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266248, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg" 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4019481913, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264846, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml" 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4055237784, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265148, 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml" 2024-05-10T21:18:24.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4057288595, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631116, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml" 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4088705810, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264597, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg" 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.677 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4091189815, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266338, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg" 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4096325839, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266332, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg" 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4096613248, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265953, 2024-05-10T21:18:24.678 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml" 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4109755616, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631020, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml" 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4121392551, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631031, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml" 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.679 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4129883636, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266365, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg" 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4135802011, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265143, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg" 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.680 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4136167962, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631019, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml" 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4136309626, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266302, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg" 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4162838436, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631140, 2024-05-10T21:18:24.681 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml" 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4180422718, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264616, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg" 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4181619219, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264851, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml" 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.682 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4193455841, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264735, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml" 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4194666107, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511630962, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml" 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.683 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4204736345, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631100, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml" 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4215387765, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266136, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg" 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4243382375, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023265966, 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml" 2024-05-10T21:18:24.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4248773657, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 1099511631023, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml" 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4250746770, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264701, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml" 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.685 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4257512955, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266011, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml" 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4265832928, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264899, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg" 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4269066943, 2024-05-10T21:18:24.686 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023264721, 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml" 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: "damage_type": "backtrace", 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4279439649, 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 2199023266344, 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: "path": "/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg" 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:24.687 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-05-10T21:18:25.329 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:25 smithi050 ceph-mon[29546]: pgmap v748: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-05-10T21:18:25.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:25 smithi092 ceph-mon[34415]: pgmap v748: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-05-10T21:18:25.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:25 smithi142 ceph-mon[34846]: pgmap v748: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 280 op/s 2024-05-10T21:18:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 -- 172.21.15.50:0/3671555340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 msgr2=0x7fe9ec1056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 --2- 172.21.15.50:0/3671555340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec1056a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009990 tx=0x7fe9dc009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 -- 172.21.15.50:0/3671555340 shutdown_connections 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 --2- 172.21.15.50:0/3671555340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 --2- 172.21.15.50:0/3671555340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec1056a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 --2- 172.21.15.50:0/3671555340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 -- 172.21.15.50:0/3671555340 >> 172.21.15.50:0/3671555340 conn(0x7fe9ec078ac0 msgr2=0x7fe9ec078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 -- 172.21.15.50:0/3671555340 shutdown_connections 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.160+0000 7fe9eb200700 1 -- 172.21.15.50:0/3671555340 wait complete. 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 Processor -- start 2024-05-10T21:18:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 -- start start 2024-05-10T21:18:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1142a0 con 0x7fe9ec10e700 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec114120 con 0x7fe9ec105be0 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.162+0000 7fe9eb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec114420 con 0x7fe9ec105290 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9ea800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9ea800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59484/0 (socket says 172.21.15.50:59484) 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41576/0 (socket says 172.21.15.50:41576) 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9ea800700 1 -- 172.21.15.50:0/2895575710 learned_addr learned my addr 172.21.15.50:0/2895575710 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50878/0 (socket says 172.21.15.50:50878) 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 -- 172.21.15.50:0/2895575710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 msgr2=0x7fe9ec19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 -- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 msgr2=0x7fe9ec1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e1400700 1 -- 172.21.15.50:0/2895575710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec1a1d20 con 0x7fe9ec105be0 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9e9e00700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.163+0000 7fe9ea800700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9e1400700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400ef10 tx=0x7fe9d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400c9c0 con 0x7fe9ec105be0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9d4019070 con 0x7fe9ec105be0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4014480 con 0x7fe9ec105be0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec1a1fb0 con 0x7fe9ec105be0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.164+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1a24a0 con 0x7fe9ec105be0 2024-05-10T21:18:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.166+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec0512c0 con 0x7fe9ec105be0 2024-05-10T21:18:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.169+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe9d4012050 con 0x7fe9ec105be0 2024-05-10T21:18:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.170+0000 7fe9e3400700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 0x7fe9d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.170+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fe9d40e2870 con 0x7fe9ec105be0 2024-05-10T21:18:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.171+0000 7fe9e9e00700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 0x7fe9d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.171+0000 7fe9e9e00700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 0x7fe9d00b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b2e0 tx=0x7fe9dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.172+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9d4010040 con 0x7fe9ec105be0 2024-05-10T21:18:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.469+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ec19e1b0 con 0x7fe9ec105be0 2024-05-10T21:18:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.470+0000 7fe9e3400700 1 -- 172.21.15.50:0/2895575710 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7fe9d40aa560 con 0x7fe9ec105be0 2024-05-10T21:18:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:18:26.467 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:17:38.867551+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":206,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26813":{"gid":26813,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1239523786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1239523786},{"type":"v1","addr":"172.21.15.92:6837","nonce":1239523786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:18:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.473+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 msgr2=0x7fe9d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.473+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 0x7fe9d00b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b2e0 tx=0x7fe9dc005fd0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.473+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 msgr2=0x7fe9ec19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.473+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400ef10 tx=0x7fe9d400c3a0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 shutdown_connections 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe9d00b2880 0x7fe9d00b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ec10e700 0x7fe9ec1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ec105290 0x7fe9ec19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 --2- 172.21.15.50:0/2895575710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec105be0 0x7fe9ec19d3f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 >> 172.21.15.50:0/2895575710 conn(0x7fe9ec078ac0 msgr2=0x7fe9ec0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 shutdown_connections 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:26.474+0000 7fe9eb200700 1 -- 172.21.15.50:0/2895575710 wait complete. 2024-05-10T21:18:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 40 2024-05-10T21:18:27.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T21:18:27.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:27 smithi050 ceph-mon[29546]: pgmap v749: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-05-10T21:18:27.134 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:27 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2895575710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:27.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:27 smithi092 ceph-mon[34415]: pgmap v749: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-05-10T21:18:27.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:27 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2895575710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:27.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:27 smithi142 ceph-mon[34846]: pgmap v749: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-05-10T21:18:27.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:27 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2895575710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:18:27.696 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24539 quiesce db --cancel --all' 2024-05-10T21:18:27.696 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- timeout 75 ceph tell mds.24539 quiesce db --cancel --all 2024-05-10T21:18:27.868 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:29 smithi050 ceph-mon[29546]: pgmap v750: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 643 KiB/s wr, 201 op/s 2024-05-10T21:18:29.309 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:29.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:29 smithi092 ceph-mon[34415]: pgmap v750: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 643 KiB/s wr, 201 op/s 2024-05-10T21:18:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:29 smithi142 ceph-mon[34846]: pgmap v750: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 643 KiB/s wr, 201 op/s 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.818+0000 7fb466800700 1 -- 172.21.15.50:0/1376572674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 msgr2=0x7fb460114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.818+0000 7fb466800700 1 --2- 172.21.15.50:0/1376572674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb460114e20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009990 tx=0x7fb44c009ca0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 -- 172.21.15.50:0/1376572674 shutdown_connections 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 --2- 172.21.15.50:0/1376572674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb460117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 --2- 172.21.15.50:0/1376572674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb460077080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 --2- 172.21.15.50:0/1376572674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb460114e20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 -- 172.21.15.50:0/1376572674 >> 172.21.15.50:0/1376572674 conn(0x7fb4600ffef0 msgr2=0x7fb460102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 -- 172.21.15.50:0/1376572674 shutdown_connections 2024-05-10T21:18:30.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.819+0000 7fb466800700 1 -- 172.21.15.50:0/1376572674 wait complete. 2024-05-10T21:18:30.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.821+0000 7fb466800700 1 Processor -- start 2024-05-10T21:18:30.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.821+0000 7fb466800700 1 -- start start 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.821+0000 7fb466800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb4601a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.821+0000 7fb466800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb4601a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.821+0000 7fb466800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb4601a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb466800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb46011aab0 con 0x7fb460115360 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb466800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb46011a930 con 0x7fb4600775c0 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb466800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb46011ac30 con 0x7fb460076c70 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb4601a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb4601a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb4601a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41594/0 (socket says 172.21.15.50:41594) 2024-05-10T21:18:30.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45f400700 1 -- 172.21.15.50:0/3330725451 learned_addr learned my addr 172.21.15.50:0/3330725451 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:30.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45fe00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb4601a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 -- 172.21.15.50:0/3330725451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 msgr2=0x7fb4601a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb4601a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 -- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 msgr2=0x7fb4601a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb4601a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45ea00700 1 -- 172.21.15.50:0/3330725451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4601a6350 con 0x7fb4600775c0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.822+0000 7fb45fe00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb4601a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb45ea00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb4601a1a20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098e0 tx=0x7fb44c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb457e00700 1 -- 172.21.15.50:0/3330725451 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c027070 con 0x7fb4600775c0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4601a65e0 con 0x7fb4600775c0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601adf60 con 0x7fb4600775c0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb457e00700 1 -- 172.21.15.50:0/3330725451 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c022470 con 0x7fb4600775c0 2024-05-10T21:18:30.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.823+0000 7fb457e00700 1 -- 172.21.15.50:0/3330725451 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c017650 con 0x7fb4600775c0 2024-05-10T21:18:30.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.829+0000 7fb457e00700 1 -- 172.21.15.50:0/3330725451 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb44c02a070 con 0x7fb4600775c0 2024-05-10T21:18:30.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.830+0000 7fb47543db80 1 Processor -- start 2024-05-10T21:18:30.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.830+0000 7fb47543db80 1 -- start start 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.830+0000 7fb47543db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.830+0000 7fb47543db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81bdf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.830+0000 7fb47543db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81bdfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb47543db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a81b01a20 con 0x561a81bdf9f0 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb47543db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561a81a9f760 con 0x561a81a61e40 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb47543db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a81afd9e0 con 0x561a81be5830 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81bdfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb45fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb457e00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 0x7fb4440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb45fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50890/0 (socket says 172.21.15.50:50890) 2024-05-10T21:18:30.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb45fe00700 1 -- 172.21.15.50:0/476823401 learned_addr learned my addr 172.21.15.50:0/476823401 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:30.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.831+0000 7fb45f400700 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81bdf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 -- 172.21.15.50:0/476823401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 msgr2=0x561a81be52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 -- 172.21.15.50:0/476823401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 msgr2=0x561a81bdfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81bdfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb457e00700 1 -- 172.21.15.50:0/3330725451 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fb44c0e2ef0 con 0x7fb4600775c0 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 -- 172.21.15.50:0/476823401 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a81a4e8d0 con 0x561a81be5830 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45fe00700 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45fe00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 0x7fb4440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb45f400700 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81bdf4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb450009d10 tx=0x7fb450011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b3d2ac68c9e8d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb454c00700 1 -- 172.21.15.50:0/476823401 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450004250 con 0x561a81be5830 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb454c00700 1 -- 172.21.15.50:0/476823401 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4500043f0 con 0x561a81be5830 2024-05-10T21:18:30.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.832+0000 7fb454c00700 1 -- 172.21.15.50:0/476823401 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450005580 con 0x561a81be5830 2024-05-10T21:18:30.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.833+0000 7fb47543db80 1 -- 172.21.15.50:0/476823401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 msgr2=0x561a81bdf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.833+0000 7fb47543db80 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81bdf4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb450009d10 tx=0x7fb450011720 comp rx=0 tx=0).stop 2024-05-10T21:18:30.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.833+0000 7fb45fe00700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 0x7fb4440b4e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4580076a0 tx=0x7fb458009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 -- 172.21.15.50:0/476823401 shutdown_connections 2024-05-10T21:18:30.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81be52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81bdfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 --2- 172.21.15.50:0/476823401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81bdf4b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 -- 172.21.15.50:0/476823401 >> 172.21.15.50:0/476823401 conn(0x561a81aa6f70 msgr2=0x561a81bda2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 -- 172.21.15.50:0/476823401 shutdown_connections 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 -- 172.21.15.50:0/476823401 wait complete. 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.834+0000 7fb47543db80 1 -- start start 2024-05-10T21:18:30.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 Processor -- start 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81be8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a81bec310 con 0x561a81a61e40 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561a81a67b20 con 0x561a81bdf9f0 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a81bec490 con 0x561a81be5830 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.835+0000 7fb47543db80 10 client.0 _ll_register_callbacks cb 0x561a81aa8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50900/0 (socket says 172.21.15.50:50900) 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 -- 172.21.15.50:0/2215184983 learned_addr learned my addr 172.21.15.50:0/2215184983 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:30.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45f400700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81be8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45fe00700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 -- 172.21.15.50:0/2215184983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 msgr2=0x561a81be9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 -- 172.21.15.50:0/2215184983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 msgr2=0x561a81be8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81be8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 -- 172.21.15.50:0/2215184983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a81ac7420 con 0x561a81bdf9f0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45fe00700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0e09c0 tx=0x7fb44c022be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca4009033b380e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.836+0000 7fb449600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0aa080 con 0x561a81bdf9f0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fb44c0094b0 con 0x561a81bdf9f0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4500043f0 con 0x561a81bdf9f0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb44c0aac70 con 0x561a81bdf9f0 2024-05-10T21:18:30.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb47543db80 10 client.26993 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561a81bf6e20 con 0x561a81bdf9f0 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4500043f0 con 0x561a81bdf9f0 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.837+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fb44c0aa860 con 0x561a81bdf9f0 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 10 client.26993 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 10 client.26993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 20 client.26993 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81bf9390 0x561a81bfb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 4 client.26993 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a81bac1c0 con 0x561a81bf9390 2024-05-10T21:18:30.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.838+0000 7fb45f400700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81bf9390 0x561a81bfb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.839+0000 7fb45f400700 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81bf9390 0x561a81bfb7e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45fb5f57dc9ef43 server_cookie=e85cd3a32b255699 in_seq=0 out_seq=0 2024-05-10T21:18:30.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.839+0000 7fb449600700 10 client.26993 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:30.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb449600700 1 -- 172.21.15.50:0/2215184983 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a81bac1c0 con 0x561a81bf9390 2024-05-10T21:18:30.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb449600700 10 client.26993 handle_command_reply: tid=0 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 client.26993 shutdown 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 msgr2=0x561a81be94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0e09c0 tx=0x7fb44c022be0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 shutdown_connections 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81bf9390 0x561a81bfb7e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81be8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 --2- 172.21.15.50:0/2215184983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81be94d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 >> 172.21.15.50:0/2215184983 conn(0x561a81aa6f70 msgr2=0x561a81bde4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb449600700 0 client.26993 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 shutdown_connections 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 1 -- 172.21.15.50:0/2215184983 wait complete. 2024-05-10T21:18:30.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.840+0000 7fb47543db80 20 client.26993 trim_cache size 0 max 16384 2024-05-10T21:18:30.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.857+0000 7fb47543db80 1 Processor -- start 2024-05-10T21:18:30.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.857+0000 7fb47543db80 1 -- start start 2024-05-10T21:18:30.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81bdf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b7b80 con 0x561a81bdf9f0 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b5350 con 0x561a81a61e40 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb47543db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b31e0 con 0x561a81be5830 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59540/0 (socket says 172.21.15.50:59540) 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45ea00700 1 -- 172.21.15.50:0/1628489421 learned_addr learned my addr 172.21.15.50:0/1628489421 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45f400700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81bdf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45fe00700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45ea00700 1 -- 172.21.15.50:0/1628489421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 msgr2=0x561a81be33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.858+0000 7fb45ea00700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45ea00700 1 -- 172.21.15.50:0/1628489421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 msgr2=0x561a81bdf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45ea00700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81bdf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45ea00700 1 -- 172.21.15.50:0/1628489421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a81be82d0 con 0x561a81bdf9f0 2024-05-10T21:18:30.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45f400700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81bdf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:30.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45fe00700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:30.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb45ea00700 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0aabd0 tx=0x7fb44c0a9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ae2cdf1950c80e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb44be00700 1 -- 172.21.15.50:0/1628489421 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0a98c0 con 0x561a81bdf9f0 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb44be00700 1 -- 172.21.15.50:0/1628489421 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c0aa570 con 0x561a81bdf9f0 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.859+0000 7fb44be00700 1 -- 172.21.15.50:0/1628489421 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c005100 con 0x561a81bdf9f0 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 -- 172.21.15.50:0/1628489421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 msgr2=0x561a81be9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0aabd0 tx=0x7fb44c0a9f70 comp rx=0 tx=0).stop 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 -- 172.21.15.50:0/1628489421 shutdown_connections 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81a61e40 0x561a81bdf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81be33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 --2- 172.21.15.50:0/1628489421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81bdf9f0 0x561a81be9850 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 -- 172.21.15.50:0/1628489421 >> 172.21.15.50:0/1628489421 conn(0x561a81aa6f70 msgr2=0x561a81be2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 -- 172.21.15.50:0/1628489421 shutdown_connections 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.860+0000 7fb47543db80 1 -- 172.21.15.50:0/1628489421 wait complete. 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.861+0000 7fb47543db80 1 -- start start 2024-05-10T21:18:30.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.861+0000 7fb47543db80 1 Processor -- start 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81bf1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81c006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b3830 con 0x561a81a61e40 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b4c80 con 0x561a81bdf9f0 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b3ee0 con 0x561a81be5830 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb47543db80 10 client.0 _ll_register_callbacks cb 0x561a81aa8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb45ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50904/0 (socket says 172.21.15.50:50904) 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb45ea00700 1 -- 172.21.15.50:0/2388594913 learned_addr learned my addr 172.21.15.50:0/2388594913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb45f400700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81bf1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.862+0000 7fb45fe00700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81c006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 -- 172.21.15.50:0/2388594913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 msgr2=0x561a81c006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81c006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 -- 172.21.15.50:0/2388594913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 msgr2=0x561a81bf1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81bf1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 -- 172.21.15.50:0/2388594913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a81c00bf0 con 0x561a81bdf9f0 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb45ea00700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0a6b10 tx=0x7fb44c0b3400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7e81184ea148d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb449600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:30.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0ae920 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fb44c0aa7d0 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c0aa570 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb44c0aa9d0 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb47543db80 10 client.26999 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.863+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561a81c00bf0 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c0aa570 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fb44c0a5ee0 con 0x561a81bdf9f0 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 10 client.26999 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 10 client.26999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 20 client.26999 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81c638f0 0x561a81c65d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 4 client.26999 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T21:18:30.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a81b0f400 con 0x561a81c638f0 2024-05-10T21:18:30.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.864+0000 7fb45f400700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81c638f0 0x561a81c65d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.865+0000 7fb45f400700 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81c638f0 0x561a81c65d10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525d577e92c26126 server_cookie=81eb2d0d660afd90 in_seq=0 out_seq=0 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.865+0000 7fb449600700 10 client.26999 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.865+0000 7fb449600700 1 -- 172.21.15.50:0/2388594913 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a81b0f400 con 0x561a81c638f0 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.865+0000 7fb449600700 10 client.26999 handle_command_reply: tid=0 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.865+0000 7fb47543db80 1 client.26999 shutdown 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 msgr2=0x561a81c00170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0a6b10 tx=0x7fb44c0b3400 comp rx=0 tx=0).stop 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 shutdown_connections 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x561a81c638f0 0x561a81c65d10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a81be5830 0x561a81c006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a81a61e40 0x561a81bf1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 --2- 172.21.15.50:0/2388594913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a81bdf9f0 0x561a81c00170 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb449600700 0 client.26999 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 >> 172.21.15.50:0/2388594913 conn(0x561a81aa6f70 msgr2=0x561a81bd40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 shutdown_connections 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 1 -- 172.21.15.50:0/2388594913 wait complete. 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.866+0000 7fb47543db80 20 client.26999 trim_cache size 0 max 16384 2024-05-10T21:18:30.865 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-10T21:18:30.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 msgr2=0x7fb4440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 0x7fb4440b4e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4580076a0 tx=0x7fb458009040 comp rx=0 tx=0).stop 2024-05-10T21:18:30.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 msgr2=0x7fb4601a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:30.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb4601a1a20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098e0 tx=0x7fb44c0057c0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 shutdown_connections 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fb4440b2900 0x7fb4440b4e10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460115360 0x7fb4601a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb460076c70 0x7fb4601a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 --2- 172.21.15.50:0/3330725451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4600775c0 0x7fb4601a1a20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 >> 172.21.15.50:0/3330725451 conn(0x7fb4600ffef0 msgr2=0x7fb4601006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 shutdown_connections 2024-05-10T21:18:30.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:30.867+0000 7fb466800700 1 -- 172.21.15.50:0/3330725451 wait complete. 2024-05-10T21:18:31.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:31 smithi092 ceph-mon[34415]: pgmap v751: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 628 KiB/s wr, 142 op/s 2024-05-10T21:18:31.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:31 smithi142 ceph-mon[34846]: pgmap v751: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 628 KiB/s wr, 142 op/s 2024-05-10T21:18:31.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:31 smithi050 ceph-mon[29546]: pgmap v751: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 628 KiB/s wr, 142 op/s 2024-05-10T21:18:31.989 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 27251176, 'ino': 2199023266368, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 41781722, 'ino': 2199023264776, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-38.xml'}, {'damage_type': 'backtrace', 'id': 51077255, 'ino': 2199023264600, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 68773459, 'ino': 2199023264171, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-75.xml'}, {'damage_type': 'backtrace', 'id': 70453880, 'ino': 2199023264816, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-18.xml'}, {'damage_type': 'backtrace', 'id': 79329381, 'ino': 2199023266100, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-5.xml'}, {'damage_type': 'backtrace', 'id': 104176075, 'ino': 2199023264826, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-13.xml'}, {'damage_type': 'backtrace', 'id': 108379976, 'ino': 1099511631119, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-37.xml'}, {'damage_type': 'backtrace', 'id': 117266632, 'ino': 1099511631033, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-20.xml'}, {'damage_type': 'backtrace', 'id': 122773241, 'ino': 2199023264149, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-85.xml'}, {'damage_type': 'backtrace', 'id': 125915668, 'ino': 1099511633027, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-49.xml'}, {'damage_type': 'backtrace', 'id': 132772991, 'ino': 1099511630999, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-53.xml'}, {'damage_type': 'backtrace', 'id': 133488554, 'ino': 2199023264932, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 134284715, 'ino': 2199023264148, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-69.xml'}, {'damage_type': 'backtrace', 'id': 144446297, 'ino': 2199023266127, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-73.jpg'}, {'damage_type': 'backtrace', 'id': 152945887, 'ino': 1099511630992, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-60.xml'}, {'damage_type': 'backtrace', 'id': 153471331, 'ino': 2199023264814, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-19.xml'}, {'damage_type': 'backtrace', 'id': 153767616, 'ino': 1099511631141, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-18.xml'}, {'damage_type': 'backtrace', 'id': 166668602, 'ino': 2199023264871, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 172343981, 'ino': 2199023266025, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-25.xml'}, {'damage_type': 'backtrace', 'id': 172899425, 'ino': 2199023266130, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-72.jpg'}, {'damage_type': 'backtrace', 'id': 182392436, 'ino': 1099511632166, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-4.xml'}, {'damage_type': 'backtrace', 'id': 200445241, 'ino': 1099511631121, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 206968987, 'ino': 2199023270879, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-96.xml'}, {'damage_type': 'backtrace', 'id': 219471537, 'ino': 2199023266308, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 221035555, 'ino': 1099511631034, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-19.xml'}, {'damage_type': 'backtrace', 'id': 221643951, 'ino': 1099511631144, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 226673575, 'ino': 2199023266281, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 232040564, 'ino': 2199023265960, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-46.xml'}, {'damage_type': 'backtrace', 'id': 234127358, 'ino': 2199023264562, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 236578561, 'ino': 2199023266189, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-54.jpg'}, {'damage_type': 'backtrace', 'id': 249690468, 'ino': 2199023266084, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-9.xml'}, {'damage_type': 'backtrace', 'id': 251265029, 'ino': 2199023265951, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-49.xml'}, {'damage_type': 'backtrace', 'id': 259342464, 'ino': 2199023264529, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 267529433, 'ino': 1099511630968, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-82.xml'}, {'damage_type': 'backtrace', 'id': 296462030, 'ino': 1099511631016, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-36.xml'}, {'damage_type': 'backtrace', 'id': 299179713, 'ino': 2199023264272, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-34.xml'}, {'damage_type': 'backtrace', 'id': 301577335, 'ino': 2199023265943, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-51.xml'}, {'damage_type': 'backtrace', 'id': 305158190, 'ino': 2199023266076, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-11.xml'}, {'damage_type': 'backtrace', 'id': 321310393, 'ino': 1099511631237, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 325684567, 'ino': 2199023264660, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 345492563, 'ino': 2199023264213, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-62.xml'}, {'damage_type': 'backtrace', 'id': 352322637, 'ino': 2199023264960, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 352599097, 'ino': 2199023264967, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-56.xml'}, {'damage_type': 'backtrace', 'id': 363645441, 'ino': 2199023264202, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-67.xml'}, {'damage_type': 'backtrace', 'id': 365266567, 'ino': 2199023264271, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-44.xml'}, {'damage_type': 'backtrace', 'id': 382467249, 'ino': 2199023266068, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-13.xml'}, {'damage_type': 'backtrace', 'id': 394714925, 'ino': 2199023264731, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-55.xml'}, {'damage_type': 'backtrace', 'id': 400868388, 'ino': 1099511630970, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-80.xml'}, {'damage_type': 'backtrace', 'id': 402650920, 'ino': 2199023266290, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 403741959, 'ino': 2199023266115, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-77.jpg'}, {'damage_type': 'backtrace', 'id': 405245706, 'ino': 1099511632169, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-1.xml'}, {'damage_type': 'backtrace', 'id': 424140301, 'ino': 1099511630998, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-54.xml'}, {'damage_type': 'backtrace', 'id': 442998730, 'ino': 1099511631151, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 458641551, 'ino': 2199023264802, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-25.xml'}, {'damage_type': 'backtrace', 'id': 471303893, 'ino': 1099511630982, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-70.xml'}, {'damage_type': 'backtrace', 'id': 485265787, 'ino': 1099511631103, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-53.xml'}, {'damage_type': 'backtrace', 'id': 498219331, 'ino': 2199023264790, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-31.xml'}, {'damage_type': 'backtrace', 'id': 501364449, 'ino': 1099511631283, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-82.xml'}, {'damage_type': 'backtrace', 'id': 516669425, 'ino': 2199023264556, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-39.jpg'}, {'damage_type': 'backtrace', 'id': 525251196, 'ino': 2199023264690, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 529865820, 'ino': 2199023264229, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-51.xml'}, {'damage_type': 'backtrace', 'id': 531395852, 'ino': 1099511631148, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-13.xml'}, {'damage_type': 'backtrace', 'id': 538682343, 'ino': 2199023266329, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 547333965, 'ino': 1099511631160, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-5.xml'}, {'damage_type': 'backtrace', 'id': 564974996, 'ino': 2199023266347, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 575850797, 'ino': 2199023264719, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-59.xml'}, {'damage_type': 'backtrace', 'id': 578784361, 'ino': 2199023266181, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-55.jpg'}, {'damage_type': 'backtrace', 'id': 581979204, 'ino': 1099511632162, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-8.xml'}, {'damage_type': 'backtrace', 'id': 601493855, 'ino': 1099511630973, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-77.xml'}, {'damage_type': 'backtrace', 'id': 626827740, 'ino': 1099511630985, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-67.xml'}, {'damage_type': 'backtrace', 'id': 630201590, 'ino': 1099511630974, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-66.xml'}, {'damage_type': 'backtrace', 'id': 632790521, 'ino': 1099511631017, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 633335506, 'ino': 2199023264127, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-78.xml'}, {'damage_type': 'backtrace', 'id': 635807912, 'ino': 2199023266326, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 642442537, 'ino': 2199023264168, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-77.xml'}, {'damage_type': 'backtrace', 'id': 642983058, 'ino': 2199023264503, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 666032959, 'ino': 2199023266064, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-14.xml'}, {'damage_type': 'backtrace', 'id': 672449157, 'ino': 2199023264788, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-32.xml'}, {'damage_type': 'backtrace', 'id': 684607469, 'ino': 2199023264350, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-17.xml'}, {'damage_type': 'backtrace', 'id': 691919846, 'ino': 2199023264579, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 700421982, 'ino': 1099511632163, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 730195983, 'ino': 1099511631125, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 739211789, 'ino': 2199023264247, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-45.xml'}, {'damage_type': 'backtrace', 'id': 740840225, 'ino': 1099511630963, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-87.xml'}, {'damage_type': 'backtrace', 'id': 743339957, 'ino': 1099511630964, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-86.xml'}, {'damage_type': 'backtrace', 'id': 745027693, 'ino': 2199023266350, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 751677761, 'ino': 2199023266039, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-21.xml'}, {'damage_type': 'backtrace', 'id': 768844333, 'ino': 2199023264210, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-63.xml'}, {'damage_type': 'backtrace', 'id': 782638177, 'ino': 1099511631219, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-62.xml'}, {'damage_type': 'backtrace', 'id': 801404940, 'ino': 1099511631013, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-39.xml'}, {'damage_type': 'backtrace', 'id': 815482504, 'ino': 2199023266060, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-15.xml'}, {'damage_type': 'backtrace', 'id': 816911223, 'ino': 2199023264796, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-28.xml'}, {'damage_type': 'backtrace', 'id': 822557670, 'ino': 2199023264333, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-23.xml'}, {'damage_type': 'backtrace', 'id': 822687363, 'ino': 2199023266213, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-50.jpg'}, {'damage_type': 'backtrace', 'id': 846939547, 'ino': 2199023264391, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-1.xml'}, {'damage_type': 'backtrace', 'id': 848703443, 'ino': 1099511630995, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-57.xml'}, {'damage_type': 'backtrace', 'id': 849786807, 'ino': 2199023264208, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-64.xml'}, {'damage_type': 'backtrace', 'id': 852223549, 'ino': 1099511631120, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-36.xml'}, {'damage_type': 'backtrace', 'id': 852745254, 'ino': 1099511631241, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-17.xml'}, {'damage_type': 'backtrace', 'id': 863342259, 'ino': 2199023266145, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 902622058, 'ino': 2199023264207, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-65.xml'}, {'damage_type': 'backtrace', 'id': 904920638, 'ino': 2199023264404, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 916113510, 'ino': 2199023264830, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-11.xml'}, {'damage_type': 'backtrace', 'id': 916177110, 'ino': 2199023264681, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 923841343, 'ino': 1099511630996, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-56.xml'}, {'damage_type': 'backtrace', 'id': 926953038, 'ino': 1099511631154, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 929868263, 'ino': 2199023265921, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-57.xml'}, {'damage_type': 'backtrace', 'id': 941980520, 'ino': 1099511631111, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 952233161, 'ino': 1099511630971, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-79.xml'}, {'damage_type': 'backtrace', 'id': 967572611, 'ino': 2199023264416, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-70.jpg'}, {'damage_type': 'backtrace', 'id': 968018449, 'ino': 2199023266133, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-71.jpg'}, {'damage_type': 'backtrace', 'id': 968417754, 'ino': 2199023264592, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 974071158, 'ino': 1099511631145, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-16.xml'}, {'damage_type': 'backtrace', 'id': 989323725, 'ino': 2199023265956, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-47.xml'}, {'damage_type': 'backtrace', 'id': 996569383, 'ino': 2199023265987, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-37.xml'}, {'damage_type': 'backtrace', 'id': 998482170, 'ino': 1099511631236, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-24.xml'}, {'damage_type': 'backtrace', 'id': 1022287994, 'ino': 2199023266178, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-56.jpg'}, {'damage_type': 'backtrace', 'id': 1025968701, 'ino': 2199023265918, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-58.xml'}, {'damage_type': 'backtrace', 'id': 1026303607, 'ino': 2199023265932, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-54.xml'}, {'damage_type': 'backtrace', 'id': 1033869196, 'ino': 2199023266032, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-23.xml'}, {'damage_type': 'backtrace', 'id': 1036567735, 'ino': 2199023264810, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-21.xml'}, {'damage_type': 'backtrace', 'id': 1038771223, 'ino': 2199023264804, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-24.xml'}, {'damage_type': 'backtrace', 'id': 1051386037, 'ino': 2199023265962, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-45.xml'}, {'damage_type': 'backtrace', 'id': 1054995731, 'ino': 2199023264294, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-36.xml'}, {'damage_type': 'backtrace', 'id': 1071144628, 'ino': 1099511630993, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-59.xml'}, {'damage_type': 'backtrace', 'id': 1084531862, 'ino': 2199023265947, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-50.xml'}, {'damage_type': 'backtrace', 'id': 1109560301, 'ino': 1099511631164, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-1.xml'}, {'damage_type': 'backtrace', 'id': 1116059498, 'ino': 2199023265996, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-34.xml'}, {'damage_type': 'backtrace', 'id': 1134077563, 'ino': 1099511631108, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 1142650262, 'ino': 1099511630984, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-68.xml'}, {'damage_type': 'backtrace', 'id': 1145967154, 'ino': 1099511631029, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-24.xml'}, {'damage_type': 'backtrace', 'id': 1164255611, 'ino': 2199023266118, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-76.jpg'}, {'damage_type': 'backtrace', 'id': 1164801078, 'ino': 2199023266275, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 1167818205, 'ino': 1099511631132, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-25.xml'}, {'damage_type': 'backtrace', 'id': 1170625093, 'ino': 1099511630981, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-71.xml'}, {'damage_type': 'backtrace', 'id': 1174411399, 'ino': 2199023266028, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-24.xml'}, {'damage_type': 'backtrace', 'id': 1176058050, 'ino': 2199023264764, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-44.xml'}, {'damage_type': 'backtrace', 'id': 1183368398, 'ino': 1099511630961, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-89.xml'}, {'damage_type': 'backtrace', 'id': 1184045967, 'ino': 2199023264532, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 1190749089, 'ino': 1099511631014, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 1191135154, 'ino': 2199023264853, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 1199226691, 'ino': 2199023266106, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-3.xml'}, {'damage_type': 'backtrace', 'id': 1201884609, 'ino': 2199023266022, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-26.xml'}, {'damage_type': 'backtrace', 'id': 1202467734, 'ino': 2199023265972, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-42.xml'}, {'damage_type': 'backtrace', 'id': 1241967127, 'ino': 1099511630977, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-75.xml'}, {'damage_type': 'backtrace', 'id': 1257462865, 'ino': 2199023266121, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-75.jpg'}, {'damage_type': 'backtrace', 'id': 1277205586, 'ino': 1099511631146, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 1319573693, 'ino': 2199023264792, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-30.xml'}, {'damage_type': 'backtrace', 'id': 1344195742, 'ino': 1099511631225, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-1.xml'}, {'damage_type': 'backtrace', 'id': 1365306278, 'ino': 1099511631161, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-4.xml'}, {'damage_type': 'backtrace', 'id': 1374324174, 'ino': 2199023264142, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-88.xml'}, {'damage_type': 'backtrace', 'id': 1395129948, 'ino': 2199023264422, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-64.jpg'}, {'damage_type': 'backtrace', 'id': 1414637287, 'ino': 2199023264824, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-14.xml'}, {'damage_type': 'backtrace', 'id': 1435093493, 'ino': 2199023264768, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-42.xml'}, {'damage_type': 'backtrace', 'id': 1439100688, 'ino': 2199023266243, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-44.jpg'}, {'damage_type': 'backtrace', 'id': 1444249096, 'ino': 1099511631008, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 1459409690, 'ino': 2199023266148, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-66.jpg'}, {'damage_type': 'backtrace', 'id': 1462154511, 'ino': 1099511633699, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-93.xml'}, {'damage_type': 'backtrace', 'id': 1469172104, 'ino': 1099511631159, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 1478589130, 'ino': 2199023266253, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-42.jpg'}, {'damage_type': 'backtrace', 'id': 1482537863, 'ino': 2199023266284, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 1487841752, 'ino': 1099511632164, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 1493625125, 'ino': 1099511631010, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-42.xml'}, {'damage_type': 'backtrace', 'id': 1493937181, 'ino': 1099511631015, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-37.xml'}, {'damage_type': 'backtrace', 'id': 1501777228, 'ino': 2199023264909, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 1509349008, 'ino': 1099511631165, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/article-80.xml'}, {'damage_type': 'backtrace', 'id': 1521927933, 'ino': 2199023264468, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 1522125727, 'ino': 2199023266072, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1531103205, 'ino': 2199023264373, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-9.xml'}, {'damage_type': 'backtrace', 'id': 1568637904, 'ino': 2199023264137, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-74.xml'}, {'damage_type': 'backtrace', 'id': 1570919360, 'ino': 1099511631131, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-26.xml'}, {'damage_type': 'backtrace', 'id': 1570988358, 'ino': 2199023266018, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-27.xml'}, {'damage_type': 'backtrace', 'id': 1596237404, 'ino': 2199023266092, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-7.xml'}, {'damage_type': 'backtrace', 'id': 1604481995, 'ino': 1099511631026, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 1612709126, 'ino': 2199023264778, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-37.xml'}, {'damage_type': 'backtrace', 'id': 1624577820, 'ino': 2199023264917, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 1637187462, 'ino': 2199023264749, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-49.xml'}, {'damage_type': 'backtrace', 'id': 1642192697, 'ino': 1099511631135, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 1644841509, 'ino': 2199023264822, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-15.xml'}, {'damage_type': 'backtrace', 'id': 1653257842, 'ino': 1099511631134, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-23.xml'}, {'damage_type': 'backtrace', 'id': 1676945840, 'ino': 2199023266311, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 1690739389, 'ino': 2199023264842, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-5.xml'}, {'damage_type': 'backtrace', 'id': 1696412947, 'ino': 1099511631110, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-46.xml'}, {'damage_type': 'backtrace', 'id': 1706768814, 'ino': 2199023265910, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-60.xml'}, {'damage_type': 'backtrace', 'id': 1708166924, 'ino': 2199023264187, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-59.xml'}, {'damage_type': 'backtrace', 'id': 1709397460, 'ino': 1099511631037, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-16.xml'}, {'damage_type': 'backtrace', 'id': 1723340432, 'ino': 2199023266359, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 1729209212, 'ino': 2199023266207, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 1742366263, 'ino': 2199023264291, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-37.xml'}, {'damage_type': 'backtrace', 'id': 1746145650, 'ino': 1099511630972, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-78.xml'}, {'damage_type': 'backtrace', 'id': 1746435824, 'ino': 1099511630991, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-61.xml'}, {'damage_type': 'backtrace', 'id': 1747599049, 'ino': 2199023264786, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-33.xml'}, {'damage_type': 'backtrace', 'id': 1755123310, 'ino': 1099511631102, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-54.xml'}, {'damage_type': 'backtrace', 'id': 1755657843, 'ino': 2199023266380, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 1763656622, 'ino': 1099511632418, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-38.xml'}, {'damage_type': 'backtrace', 'id': 1766084662, 'ino': 2199023264654, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 1769186122, 'ino': 2199023266172, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-58.jpg'}, {'damage_type': 'backtrace', 'id': 1772521493, 'ino': 2199023264146, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-70.xml'}, {'damage_type': 'backtrace', 'id': 1774188245, 'ino': 1099511630988, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-64.xml'}, {'damage_type': 'backtrace', 'id': 1786265401, 'ino': 2199023264725, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-57.xml'}, {'damage_type': 'backtrace', 'id': 1794895395, 'ino': 2199023264747, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-50.xml'}, {'damage_type': 'backtrace', 'id': 1806853587, 'ino': 2199023266046, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-19.xml'}, {'damage_type': 'backtrace', 'id': 1809399859, 'ino': 2199023264286, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-39.xml'}, {'damage_type': 'backtrace', 'id': 1814809798, 'ino': 1099511631269, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-20.xml'}, {'damage_type': 'backtrace', 'id': 1826472703, 'ino': 1099511631248, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-59.xml'}, {'damage_type': 'backtrace', 'id': 1840699100, 'ino': 2199023266109, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-2.xml'}, {'damage_type': 'backtrace', 'id': 1855406871, 'ino': 2199023264836, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-8.xml'}, {'damage_type': 'backtrace', 'id': 1859033508, 'ino': 1099511631022, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-30.xml'}, {'damage_type': 'backtrace', 'id': 1862542682, 'ino': 1099511632212, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-49.xml'}, {'damage_type': 'backtrace', 'id': 1866963239, 'ino': 1099511630976, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-76.xml'}, {'damage_type': 'backtrace', 'id': 1885887634, 'ino': 2199023265984, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-38.xml'}, {'damage_type': 'backtrace', 'id': 1915409745, 'ino': 2199023266202, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 1915918834, 'ino': 2199023264818, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-17.xml'}, {'damage_type': 'backtrace', 'id': 1919108591, 'ino': 2199023264832, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-10.xml'}, {'damage_type': 'backtrace', 'id': 1922759373, 'ino': 2199023266195, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 1927064360, 'ino': 1099511631117, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-39.xml'}, {'damage_type': 'backtrace', 'id': 1929647193, 'ino': 1099511631133, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-24.xml'}, {'damage_type': 'backtrace', 'id': 1935881129, 'ino': 2199023264516, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 1943545615, 'ino': 1099511631038, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 1952759175, 'ino': 2199023264885, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 1995624755, 'ino': 1099511631139, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-20.xml'}, {'damage_type': 'backtrace', 'id': 2021739471, 'ino': 2199023264834, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-9.xml'}, {'damage_type': 'backtrace', 'id': 2021903939, 'ino': 2199023266374, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 2028221461, 'ino': 2199023266383, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 2034531360, 'ino': 1099511631041, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-12.xml'}, {'damage_type': 'backtrace', 'id': 2035037605, 'ino': 2199023264752, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-48.xml'}, {'damage_type': 'backtrace', 'id': 2035838983, 'ino': 1099511631112, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 2041598994, 'ino': 2199023264711, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-61.xml'}, {'damage_type': 'backtrace', 'id': 2042850268, 'ino': 2199023266112, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-1.xml'}, {'damage_type': 'backtrace', 'id': 2050908664, 'ino': 2199023265936, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-53.xml'}, {'damage_type': 'backtrace', 'id': 2055731661, 'ino': 1099511631279, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-14.xml'}, {'damage_type': 'backtrace', 'id': 2069355589, 'ino': 1099511631275, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-32.xml'}, {'damage_type': 'backtrace', 'id': 2069402495, 'ino': 1099511631030, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-23.xml'}, {'damage_type': 'backtrace', 'id': 2081696391, 'ino': 2199023266238, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-45.jpg'}, {'damage_type': 'backtrace', 'id': 2084421152, 'ino': 1099511631032, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-21.xml'}, {'damage_type': 'backtrace', 'id': 2085027275, 'ino': 1099511631113, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 2085186883, 'ino': 1099511631130, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 2097232361, 'ino': 2199023264517, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 2098008793, 'ino': 1099511631012, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-40.xml'}, {'damage_type': 'backtrace', 'id': 2098410436, 'ino': 2199023265979, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-40.xml'}, {'damage_type': 'backtrace', 'id': 2115206893, 'ino': 1099511632170, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/article-74.xml'}, {'damage_type': 'backtrace', 'id': 2118555814, 'ino': 2199023266353, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 2121873780, 'ino': 2199023264862, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 2124483121, 'ino': 2199023266166, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-60.jpg'}, {'damage_type': 'backtrace', 'id': 2124906236, 'ino': 2199023264755, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-47.xml'}, {'damage_type': 'backtrace', 'id': 2141597947, 'ino': 1099511630997, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-55.xml'}, {'damage_type': 'backtrace', 'id': 2147365898, 'ino': 2199023266287, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 2148916616, 'ino': 2199023264840, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-6.xml'}, {'damage_type': 'backtrace', 'id': 2160741414, 'ino': 2199023266163, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 2166586825, 'ino': 2199023264279, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-31.xml'}, {'damage_type': 'backtrace', 'id': 2174233877, 'ino': 2199023264875, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 2191319905, 'ino': 1099511631246, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-54.xml'}, {'damage_type': 'backtrace', 'id': 2199985986, 'ino': 1099511631027, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-26.xml'}, {'damage_type': 'backtrace', 'id': 2205803713, 'ino': 2199023264676, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 2210245074, 'ino': 2199023264672, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 2227623846, 'ino': 2199023266268, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-39.jpg'}, {'damage_type': 'backtrace', 'id': 2238048482, 'ino': 2199023264772, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-40.xml'}, {'damage_type': 'backtrace', 'id': 2241330335, 'ino': 2199023264650, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 2250361952, 'ino': 2199023264812, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-20.xml'}, {'damage_type': 'backtrace', 'id': 2250795648, 'ino': 2199023266139, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-69.jpg'}, {'damage_type': 'backtrace', 'id': 2260664826, 'ino': 2199023264571, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 2266058140, 'ino': 2199023266154, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-64.jpg'}, {'damage_type': 'backtrace', 'id': 2272742220, 'ino': 2199023264363, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-12.xml'}, {'damage_type': 'backtrace', 'id': 2276406971, 'ino': 2199023266320, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 2276868298, 'ino': 2199023264594, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 2285379462, 'ino': 2199023265022, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-22.xml'}, {'damage_type': 'backtrace', 'id': 2288802331, 'ino': 2199023266296, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 2289618155, 'ino': 2199023264715, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-60.xml'}, {'damage_type': 'backtrace', 'id': 2294313781, 'ino': 2199023264320, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-18.xml'}, {'damage_type': 'backtrace', 'id': 2294957632, 'ino': 2199023271811, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 2297751321, 'ino': 2199023264582, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 2309170005, 'ino': 2199023264115, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-83.xml'}, {'damage_type': 'backtrace', 'id': 2313587971, 'ino': 2199023264547, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-42.jpg'}, {'damage_type': 'backtrace', 'id': 2313879867, 'ino': 1099511630994, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-58.xml'}, {'damage_type': 'backtrace', 'id': 2314207020, 'ino': 1099511631147, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-14.xml'}, {'damage_type': 'backtrace', 'id': 2328040986, 'ino': 1099511631001, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-51.xml'}, {'damage_type': 'backtrace', 'id': 2345564899, 'ino': 2199023264386, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-3.xml'}, {'damage_type': 'backtrace', 'id': 2356253608, 'ino': 1099511631162, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 2379184693, 'ino': 2199023264926, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 2394025997, 'ino': 2199023266160, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-62.jpg'}, {'damage_type': 'backtrace', 'id': 2398140620, 'ino': 2199023267598, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-56.xml'}, {'damage_type': 'backtrace', 'id': 2416015364, 'ino': 1099511631224, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/picture-74.jpg'}, {'damage_type': 'backtrace', 'id': 2435674300, 'ino': 1099511631000, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 2439515533, 'ino': 1099511631028, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-25.xml'}, {'damage_type': 'backtrace', 'id': 2448093355, 'ino': 2199023264285, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2451237315, 'ino': 2199023264806, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-23.xml'}, {'damage_type': 'backtrace', 'id': 2455648714, 'ino': 1099511630983, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-69.xml'}, {'damage_type': 'backtrace', 'id': 2475968859, 'ino': 1099511631035, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-18.xml'}, {'damage_type': 'backtrace', 'id': 2480891551, 'ino': 1099511631021, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 2489373942, 'ino': 2199023264159, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-80.xml'}, {'damage_type': 'backtrace', 'id': 2496619841, 'ino': 2199023266157, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-63.jpg'}, {'damage_type': 'backtrace', 'id': 2501582649, 'ino': 1099511631115, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-41.xml'}, {'damage_type': 'backtrace', 'id': 2506799983, 'ino': 2199023264176, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-61.xml'}, {'damage_type': 'backtrace', 'id': 2528965882, 'ino': 1099511631009, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 2544130922, 'ino': 2199023265975, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-41.xml'}, {'damage_type': 'backtrace', 'id': 2550039224, 'ino': 1099511630959, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-91.xml'}, {'damage_type': 'backtrace', 'id': 2556142876, 'ino': 2199023266362, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 2563863309, 'ino': 2199023265998, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-33.xml'}, {'damage_type': 'backtrace', 'id': 2565055024, 'ino': 2199023266323, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 2572855861, 'ino': 2199023270764, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-51.xml'}, {'damage_type': 'backtrace', 'id': 2576504882, 'ino': 1099511631163, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 2582752363, 'ino': 2199023264574, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 2582804314, 'ino': 2199023266056, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-16.xml'}, {'damage_type': 'backtrace', 'id': 2583701166, 'ino': 2199023266088, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-8.xml'}, {'damage_type': 'backtrace', 'id': 2603050229, 'ino': 1099511631007, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 2605383714, 'ino': 1099511631042, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-11.xml'}, {'damage_type': 'backtrace', 'id': 2631900879, 'ino': 1099511630990, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-62.xml'}, {'damage_type': 'backtrace', 'id': 2632458380, 'ino': 2199023264828, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-12.xml'}, {'damage_type': 'backtrace', 'id': 2637594234, 'ino': 1099511631107, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-49.xml'}, {'damage_type': 'backtrace', 'id': 2645280923, 'ino': 2199023264939, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-36.xml'}, {'damage_type': 'backtrace', 'id': 2653538245, 'ino': 1099511631114, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-42.xml'}, {'damage_type': 'backtrace', 'id': 2664564743, 'ino': 2199023264699, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-65.xml'}, {'damage_type': 'backtrace', 'id': 2685183466, 'ino': 2199023264357, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2687518681, 'ino': 1099511631004, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 2692964840, 'ino': 2199023264904, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 2712845126, 'ino': 1099511631128, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-29.xml'}, {'damage_type': 'backtrace', 'id': 2714970370, 'ino': 2199023264800, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-26.xml'}, {'damage_type': 'backtrace', 'id': 2717761821, 'ino': 1099511631123, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-33.xml'}, {'damage_type': 'backtrace', 'id': 2718401193, 'ino': 2199023266341, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 2720394854, 'ino': 2199023264330, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-24.xml'}, {'damage_type': 'backtrace', 'id': 2725946923, 'ino': 2199023266299, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 2730895197, 'ino': 2199023264794, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2735307232, 'ino': 1099511631171, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/article-2.xml'}, {'damage_type': 'backtrace', 'id': 2743610447, 'ino': 2199023264739, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-53.xml'}, {'damage_type': 'backtrace', 'id': 2756416505, 'ino': 1099511633549, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-10.xml'}, {'damage_type': 'backtrace', 'id': 2757210753, 'ino': 2199023266317, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 2760393880, 'ino': 2199023264412, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-71.jpg'}, {'damage_type': 'backtrace', 'id': 2773348980, 'ino': 2199023266272, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-38.jpg'}, {'damage_type': 'backtrace', 'id': 2784579931, 'ino': 1099511630965, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-85.xml'}, {'damage_type': 'backtrace', 'id': 2793079428, 'ino': 2199023264782, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-35.xml'}, {'damage_type': 'backtrace', 'id': 2795872165, 'ino': 1099511631149, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-12.xml'}, {'damage_type': 'backtrace', 'id': 2795887759, 'ino': 1099511632165, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-5.xml'}, {'damage_type': 'backtrace', 'id': 2821749331, 'ino': 1099511632414, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 2824780603, 'ino': 1099511630960, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-90.xml'}, {'damage_type': 'backtrace', 'id': 2839467973, 'ino': 1099511630986, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-66.xml'}, {'damage_type': 'backtrace', 'id': 2848642525, 'ino': 1099511632161, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 2850464593, 'ino': 2199023264378, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-7.xml'}, {'damage_type': 'backtrace', 'id': 2873746459, 'ino': 2199023264849, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-2.xml'}, {'damage_type': 'backtrace', 'id': 2898236511, 'ino': 2199023264858, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 2913759311, 'ino': 2199023264844, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-4.xml'}, {'damage_type': 'backtrace', 'id': 2922769514, 'ino': 2199023264953, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 2927578888, 'ino': 2199023266233, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 2928184680, 'ino': 1099511631003, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-49.xml'}, {'damage_type': 'backtrace', 'id': 2929969976, 'ino': 2199023264963, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 2937727187, 'ino': 1099511631109, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-47.xml'}, {'damage_type': 'backtrace', 'id': 2957489322, 'ino': 1099511631018, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-34.xml'}, {'damage_type': 'backtrace', 'id': 2959725509, 'ino': 2199023264400, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-68.jpg'}, {'damage_type': 'backtrace', 'id': 2974828384, 'ino': 2199023266142, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-68.jpg'}, {'damage_type': 'backtrace', 'id': 2980347315, 'ino': 2199023264455, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-63.jpg'}, {'damage_type': 'backtrace', 'id': 2982523891, 'ino': 2199023266258, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-41.jpg'}, {'damage_type': 'backtrace', 'id': 2983065204, 'ino': 1099511630987, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-65.xml'}, {'damage_type': 'backtrace', 'id': 2988291099, 'ino': 2199023264770, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-41.xml'}, {'damage_type': 'backtrace', 'id': 2993258638, 'ino': 2199023266263, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 3001382202, 'ino': 1099511631124, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-32.xml'}, {'damage_type': 'backtrace', 'id': 3002667600, 'ino': 2199023264880, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 3002864321, 'ino': 2199023264865, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 3006357152, 'ino': 1099511631002, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 3007121177, 'ino': 1099511631122, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-34.xml'}, {'damage_type': 'backtrace', 'id': 3012342749, 'ino': 2199023266053, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-17.xml'}, {'damage_type': 'backtrace', 'id': 3035750679, 'ino': 2199023264472, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-57.jpg'}, {'damage_type': 'backtrace', 'id': 3061897753, 'ino': 1099511630958, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-92.xml'}, {'damage_type': 'backtrace', 'id': 3065256453, 'ino': 2199023264705, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-63.xml'}, {'damage_type': 'backtrace', 'id': 3075262343, 'ino': 1099511633698, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 3090534988, 'ino': 3298534884586, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 3092647506, 'ino': 2199023265991, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-36.xml'}, {'damage_type': 'backtrace', 'id': 3093479070, 'ino': 1099511631106, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 3097682004, 'ino': 2199023265982, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-39.xml'}, {'damage_type': 'backtrace', 'id': 3101157198, 'ino': 2199023266124, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-74.jpg'}, {'damage_type': 'backtrace', 'id': 3118566112, 'ino': 2199023264766, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-43.xml'}, {'damage_type': 'backtrace', 'id': 3118923750, 'ino': 1099511631245, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-23/picture-69.jpg'}, {'damage_type': 'backtrace', 'id': 3129777249, 'ino': 2199023264798, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-27.xml'}, {'damage_type': 'backtrace', 'id': 3130884427, 'ino': 1099511631118, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 3131349752, 'ino': 1099511631101, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-55.xml'}, {'damage_type': 'backtrace', 'id': 3135791406, 'ino': 2199023266293, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 3148966126, 'ino': 1099511631040, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-13.xml'}, {'damage_type': 'backtrace', 'id': 3186600691, 'ino': 2199023265017, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 3202621922, 'ino': 1099511632168, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 3215296928, 'ino': 2199023264145, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-87.xml'}, {'damage_type': 'backtrace', 'id': 3225159793, 'ino': 2199023264180, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-73.xml'}, {'damage_type': 'backtrace', 'id': 3226718057, 'ino': 2199023266305, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 3231285753, 'ino': 1099511630979, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-73.xml'}, {'damage_type': 'backtrace', 'id': 3232880188, 'ino': 2199023264308, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-22.xml'}, {'damage_type': 'backtrace', 'id': 3241076895, 'ino': 2199023266371, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 3251905756, 'ino': 2199023264741, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-52.xml'}, {'damage_type': 'backtrace', 'id': 3254050727, 'ino': 2199023264694, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-66.xml'}, {'damage_type': 'backtrace', 'id': 3267675601, 'ino': 2199023265940, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-52.xml'}, {'damage_type': 'backtrace', 'id': 3268758051, 'ino': 2199023264675, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 3298448998, 'ino': 2199023266042, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-20.xml'}, {'damage_type': 'backtrace', 'id': 3316549385, 'ino': 2199023265992, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-35.xml'}, {'damage_type': 'backtrace', 'id': 3324145931, 'ino': 2199023266314, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 3330548757, 'ino': 1099511632160, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 3337125182, 'ino': 2199023266218, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-49.jpg'}, {'damage_type': 'backtrace', 'id': 3337268944, 'ino': 2199023266228, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-47.jpg'}, {'damage_type': 'backtrace', 'id': 3390164945, 'ino': 2199023264838, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-7.xml'}, {'damage_type': 'backtrace', 'id': 3404475667, 'ino': 2199023264577, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 3406984871, 'ino': 1099511631104, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 3410264128, 'ino': 2199023271015, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 3410789920, 'ino': 2199023264589, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 3416649720, 'ino': 1099511631136, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-21.xml'}, {'damage_type': 'backtrace', 'id': 3420570030, 'ino': 1099511631058, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-65.xml'}, {'damage_type': 'backtrace', 'id': 3431890584, 'ino': 2199023264925, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 3432732370, 'ino': 2199023266004, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-31.xml'}, {'damage_type': 'backtrace', 'id': 3451240814, 'ino': 2199023264868, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 3467741241, 'ino': 1099511631011, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-41.xml'}, {'damage_type': 'backtrace', 'id': 3482873968, 'ino': 1099511631025, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 3497056240, 'ino': 1099511630980, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-72.xml'}, {'damage_type': 'backtrace', 'id': 3503388525, 'ino': 2199023265914, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-59.xml'}, {'damage_type': 'backtrace', 'id': 3508030218, 'ino': 2199023266169, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-59.jpg'}, {'damage_type': 'backtrace', 'id': 3511176430, 'ino': 2199023266175, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-57.jpg'}, {'damage_type': 'backtrace', 'id': 3513023183, 'ino': 2199023264627, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3513725190, 'ino': 2199023264234, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-49.xml'}, {'damage_type': 'backtrace', 'id': 3519371974, 'ino': 2199023266049, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-18.xml'}, {'damage_type': 'backtrace', 'id': 3546102657, 'ino': 2199023270337, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-5/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 3559124357, 'ino': 2199023264942, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 3572412572, 'ino': 2199023264205, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/article-66.xml'}, {'damage_type': 'backtrace', 'id': 3577050448, 'ino': 2199023266377, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 3579892750, 'ino': 2199023264183, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-60.xml'}, {'damage_type': 'backtrace', 'id': 3583933007, 'ino': 2199023266096, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-6.xml'}, {'damage_type': 'backtrace', 'id': 3588202795, 'ino': 1099511631036, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 3589647605, 'ino': 2199023265969, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-43.xml'}, {'damage_type': 'backtrace', 'id': 3592400505, 'ino': 2199023264855, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 3601339329, 'ino': 2199023264820, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-16.xml'}, {'damage_type': 'backtrace', 'id': 3622220189, 'ino': 1099511631129, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 3623075762, 'ino': 2199023264784, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-34.xml'}, {'damage_type': 'backtrace', 'id': 3651326218, 'ino': 2199023266223, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 3657973747, 'ino': 2199023264293, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-27.xml'}, {'damage_type': 'backtrace', 'id': 3661938753, 'ino': 2199023266008, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-30.xml'}, {'damage_type': 'backtrace', 'id': 3674528723, 'ino': 2199023266151, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-65.jpg'}, {'damage_type': 'backtrace', 'id': 3693790575, 'ino': 1099511631152, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 3699126502, 'ino': 1099511631153, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-8.xml'}, {'damage_type': 'backtrace', 'id': 3705506085, 'ino': 2199023266335, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 3711573514, 'ino': 2199023264371, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-78.jpg'}, {'damage_type': 'backtrace', 'id': 3730225899, 'ino': 1099511631039, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-14.xml'}, {'damage_type': 'backtrace', 'id': 3755756481, 'ino': 1099511631226, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/article-62.xml'}, {'damage_type': 'backtrace', 'id': 3756118653, 'ino': 2199023264482, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-55.jpg'}, {'damage_type': 'backtrace', 'id': 3765498747, 'ino': 2199023264762, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-45.xml'}, {'damage_type': 'backtrace', 'id': 3773086919, 'ino': 2199023264313, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-20.xml'}, {'damage_type': 'backtrace', 'id': 3778696910, 'ino': 2199023266278, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 3779616914, 'ino': 1099511630967, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-83.xml'}, {'damage_type': 'backtrace', 'id': 3781765392, 'ino': 2199023265929, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-55.xml'}, {'damage_type': 'backtrace', 'id': 3785970329, 'ino': 1099511630978, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-74.xml'}, {'damage_type': 'backtrace', 'id': 3787462651, 'ino': 1099511631105, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-51.xml'}, {'damage_type': 'backtrace', 'id': 3790727952, 'ino': 2199023266015, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-28.xml'}, {'damage_type': 'backtrace', 'id': 3795829643, 'ino': 2199023265925, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-56.xml'}, {'damage_type': 'backtrace', 'id': 3796639447, 'ino': 1099511631238, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-56/article-19.xml'}, {'damage_type': 'backtrace', 'id': 3810723426, 'ino': 2199023266356, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 3818700748, 'ino': 1099511631126, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-30.xml'}, {'damage_type': 'backtrace', 'id': 3828936980, 'ino': 2199023266103, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-4.xml'}, {'damage_type': 'backtrace', 'id': 3866912066, 'ino': 1099511630989, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-63.xml'}, {'damage_type': 'backtrace', 'id': 3889732721, 'ino': 1099511630966, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-84.xml'}, {'damage_type': 'backtrace', 'id': 3893005618, 'ino': 2199023264709, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-62.xml'}, {'damage_type': 'backtrace', 'id': 3898045736, 'ino': 2199023264744, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-51.xml'}, {'damage_type': 'backtrace', 'id': 3903087160, 'ino': 1099511631006, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-46.xml'}, {'damage_type': 'backtrace', 'id': 3909901693, 'ino': 1099511631005, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-47.xml'}, {'damage_type': 'backtrace', 'id': 3921199254, 'ino': 2199023264257, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-41.xml'}, {'damage_type': 'backtrace', 'id': 3940402659, 'ino': 2199023264619, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 3944637692, 'ino': 2199023264120, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/article-81.xml'}, {'damage_type': 'backtrace', 'id': 3964301865, 'ino': 2199023266035, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-22.xml'}, {'damage_type': 'backtrace', 'id': 3976098486, 'ino': 1099511630969, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-81.xml'}, {'damage_type': 'backtrace', 'id': 3978644520, 'ino': 2199023264774, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-39.xml'}, {'damage_type': 'backtrace', 'id': 3989786101, 'ino': 1099511632167, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 3996475894, 'ino': 1099511631150, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-11.xml'}, {'damage_type': 'backtrace', 'id': 4011268582, 'ino': 2199023266248, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-43.jpg'}, {'damage_type': 'backtrace', 'id': 4019481913, 'ino': 2199023264846, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-3.xml'}, {'damage_type': 'backtrace', 'id': 4055237784, 'ino': 2199023265148, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-46.xml'}, {'damage_type': 'backtrace', 'id': 4057288595, 'ino': 1099511631116, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-40.xml'}, {'damage_type': 'backtrace', 'id': 4088705810, 'ino': 2199023264597, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-67/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 4091189815, 'ino': 2199023266338, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 4096325839, 'ino': 2199023266332, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 4096613248, 'ino': 2199023265953, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-48.xml'}, {'damage_type': 'backtrace', 'id': 4109755616, 'ino': 1099511631020, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-32.xml'}, {'damage_type': 'backtrace', 'id': 4121392551, 'ino': 1099511631031, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 4129883636, 'ino': 2199023266365, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 4135802011, 'ino': 2199023265143, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-92.jpg'}, {'damage_type': 'backtrace', 'id': 4136167962, 'ino': 1099511631019, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-33.xml'}, {'damage_type': 'backtrace', 'id': 4136309626, 'ino': 2199023266302, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 4162838436, 'ino': 1099511631140, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-19.xml'}, {'damage_type': 'backtrace', 'id': 4180422718, 'ino': 2199023264616, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-68/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 4181619219, 'ino': 2199023264851, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-1.xml'}, {'damage_type': 'backtrace', 'id': 4193455841, 'ino': 2199023264735, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-54.xml'}, {'damage_type': 'backtrace', 'id': 4194666107, 'ino': 1099511630962, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-88.xml'}, {'damage_type': 'backtrace', 'id': 4204736345, 'ino': 1099511631100, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-27/comment-56.xml'}, {'damage_type': 'backtrace', 'id': 4215387765, 'ino': 2199023266136, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-70.jpg'}, {'damage_type': 'backtrace', 'id': 4243382375, 'ino': 2199023265966, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-44.xml'}, {'damage_type': 'backtrace', 'id': 4248773657, 'ino': 1099511631023, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-49/comment-29.xml'}, {'damage_type': 'backtrace', 'id': 4250746770, 'ino': 2199023264701, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-64.xml'}, {'damage_type': 'backtrace', 'id': 4257512955, 'ino': 2199023266011, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/article-29.xml'}, {'damage_type': 'backtrace', 'id': 4265832928, 'ino': 2199023264899, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 4269066943, 'ino': 2199023264721, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-71/article-58.xml'}, {'damage_type': 'backtrace', 'id': 4279439649, 'ino': 2199023266344, 'path': '/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-84/picture-14.jpg'}] 2024-05-10T21:18:31.989 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-10T21:18:32.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 -- 172.21.15.50:0/1944026566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f8108f20 msgr2=0x7f85f8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 --2- 172.21.15.50:0/1944026566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f8108f20 0x7f85f8109330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 -- 172.21.15.50:0/1944026566 shutdown_connections 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 --2- 172.21.15.50:0/1944026566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 --2- 172.21.15.50:0/1944026566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f81045f0 0x7f85f8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 --2- 172.21.15.50:0/1944026566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f8108f20 0x7f85f8109330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 -- 172.21.15.50:0/1944026566 >> 172.21.15.50:0/1944026566 conn(0x7f85f8100040 msgr2=0x7f85f8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 -- 172.21.15.50:0/1944026566 shutdown_connections 2024-05-10T21:18:32.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.626+0000 7f85fda00700 1 -- 172.21.15.50:0/1944026566 wait complete. 2024-05-10T21:18:32.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.627+0000 7f85fda00700 1 Processor -- start 2024-05-10T21:18:32.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 -- start start 2024-05-10T21:18:32.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f8108f20 0x7f85f819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81144a0 con 0x7f85f8108f20 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8114320 con 0x7f85f81045f0 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.628+0000 7f85fda00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8114620 con 0x7f85f8109870 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85ee000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f8108f20 0x7f85f819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50914/0 (socket says 172.21.15.50:50914) 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 -- 172.21.15.50:0/2216584294 learned_addr learned my addr 172.21.15.50:0/2216584294 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 -- 172.21.15.50:0/2216584294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 msgr2=0x7f85f819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 -- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f8108f20 msgr2=0x7f85f819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f8108f20 0x7f85f819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 -- 172.21.15.50:0/2216584294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f819fe80 con 0x7f85f81045f0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f7400700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.629+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85e800e9a0 tx=0x7f85e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.630+0000 7f85efe00700 1 -- 172.21.15.50:0/2216584294 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e800ca10 con 0x7f85f81045f0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.630+0000 7f85efe00700 1 -- 172.21.15.50:0/2216584294 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e800cbb0 con 0x7f85f81045f0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.630+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f81a0170 con 0x7f85f81045f0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.630+0000 7f85efe00700 1 -- 172.21.15.50:0/2216584294 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8018680 con 0x7f85f81045f0 2024-05-10T21:18:32.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.630+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f81a7a90 con 0x7f85f81045f0 2024-05-10T21:18:32.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.634+0000 7f85efe00700 1 -- 172.21.15.50:0/2216584294 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f85e8010430 con 0x7f85f81045f0 2024-05-10T21:18:32.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 Processor -- start 2024-05-10T21:18:32.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 -- start start 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d680c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676980 0x55588d683180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55588d598a90 con 0x55588d4f8eb0 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55588d5367d0 con 0x55588d676570 2024-05-10T21:18:32.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.635+0000 7f860c59ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55588d594a50 con 0x55588d676980 2024-05-10T21:18:32.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85efe00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 0x7f85dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d680c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85f7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85f7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59574/0 (socket says 172.21.15.50:59574) 2024-05-10T21:18:32.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85f7400700 1 -- 172.21.15.50:0/775781823 learned_addr learned my addr 172.21.15.50:0/775781823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85f7400700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 0x7f85dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85ee000700 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676980 0x55588d683180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.636+0000 7f85efe00700 1 -- 172.21.15.50:0/2216584294 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f85e80e2390 con 0x7f85f81045f0 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 -- 172.21.15.50:0/775781823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676980 msgr2=0x55588d683180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676980 0x55588d683180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 -- 172.21.15.50:0/775781823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 msgr2=0x55588d680c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d680c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 -- 172.21.15.50:0/775781823 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55588d4e5940 con 0x55588d4f8eb0 2024-05-10T21:18:32.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85f0013f10 tx=0x7f85f00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16379e7ae267d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85f7400700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 0x7f85dc0b4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f85f0011a10 tx=0x7f85f0018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.637+0000 7f85e7e00700 1 -- 172.21.15.50:0/775781823 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0024070 con 0x55588d4f8eb0 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.638+0000 7f85e7e00700 1 -- 172.21.15.50:0/775781823 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0013a20 con 0x55588d4f8eb0 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.638+0000 7f85e7e00700 1 -- 172.21.15.50:0/775781823 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000ed00 con 0x55588d4f8eb0 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.638+0000 7f860c59ab80 1 -- 172.21.15.50:0/775781823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 msgr2=0x55588d676030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.638+0000 7f860c59ab80 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85f0013f10 tx=0x7f85f00116c0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 -- 172.21.15.50:0/775781823 shutdown_connections 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676980 0x55588d683180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d676030 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 --2- 172.21.15.50:0/775781823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d680c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 -- 172.21.15.50:0/775781823 >> 172.21.15.50:0/775781823 conn(0x55588d537af0 msgr2=0x55588d671e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 -- 172.21.15.50:0/775781823 shutdown_connections 2024-05-10T21:18:32.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.639+0000 7f860c59ab80 1 -- 172.21.15.50:0/775781823 wait complete. 2024-05-10T21:18:32.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.640+0000 7f860c59ab80 1 -- start start 2024-05-10T21:18:32.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.640+0000 7f860c59ab80 1 Processor -- start 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.640+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d4f8eb0 0x55588d680710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d681190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55588d683ae0 con 0x55588d676980 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55588d4feb90 con 0x55588d4f8eb0 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55588d683c60 con 0x55588d676570 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f860c59ab80 10 client.0 _ll_register_callbacks cb 0x55588d52c3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d4f8eb0 0x55588d680710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41670/0 (socket says 172.21.15.50:41670) 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 -- 172.21.15.50:0/1738934755 learned_addr learned my addr 172.21.15.50:0/1738934755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:32.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85f7400700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d681190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 -- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d4f8eb0 msgr2=0x55588d680710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d4f8eb0 0x55588d680710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 -- 172.21.15.50:0/1738934755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 msgr2=0x55588d681190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d681190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.641+0000 7f85ee000700 1 -- 172.21.15.50:0/1738934755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55588d55e490 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f85ee000700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e000bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a493ee01c59046f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f85e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0003c00 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f0013a20 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f85e0003da0 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f85e00075c0 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.642+0000 7f860c59ab80 10 client.26962 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55588d6929b0 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0013a20 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f85e0011070 con 0x55588d676570 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f860c59ab80 10 client.26962 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f860c59ab80 10 client.26962 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:18:32.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f860c59ab80 20 client.26962 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.643+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d694f90 0x55588d6973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.644+0000 7f860c59ab80 4 client.26962 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.644+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55588d6430e0 con 0x55588d694f90 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.644+0000 7f85f6a00700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d694f90 0x55588d6973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.644+0000 7f85f6a00700 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d694f90 0x55588d6973e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7c35f1d586441a server_cookie=ab96986f298b9a48 in_seq=0 out_seq=0 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.644+0000 7f85e4c00700 10 client.26962 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.645+0000 7f85e4c00700 1 -- 172.21.15.50:0/1738934755 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55588d6430e0 con 0x55588d694f90 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.645+0000 7f85e4c00700 10 client.26962 handle_command_reply: tid=0 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.645+0000 7f860c59ab80 1 client.26962 shutdown 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 msgr2=0x55588d680c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e000bd50 comp rx=0 tx=0).stop 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 shutdown_connections 2024-05-10T21:18:32.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d694f90 0x55588d6973e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d4f8eb0 0x55588d680710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d681190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1738934755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d680c50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f85e4c00700 0 client.26962 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 >> 172.21.15.50:0/1738934755 conn(0x55588d537af0 msgr2=0x55588d671150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 shutdown_connections 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 1 -- 172.21.15.50:0/1738934755 wait complete. 2024-05-10T21:18:32.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.646+0000 7f860c59ab80 20 client.26962 trim_cache size 0 max 16384 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 Processor -- start 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 -- start start 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d691000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d69cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81af370 con 0x55588d676980 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81acb40 con 0x55588d676570 2024-05-10T21:18:32.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.669+0000 7f860c59ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81aa9f0 con 0x55588d4f8eb0 2024-05-10T21:18:32.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41674/0 (socket says 172.21.15.50:41674) 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d69cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 -- 172.21.15.50:0/2779454230 learned_addr learned my addr 172.21.15.50:0/2779454230 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85ee000700 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d691000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 -- 172.21.15.50:0/2779454230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 msgr2=0x55588d691000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d691000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 -- 172.21.15.50:0/2779454230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 msgr2=0x55588d69cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d69cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 -- 172.21.15.50:0/2779454230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55588d669900 con 0x55588d4f8eb0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85ee000700 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d691000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.670+0000 7f85f6a00700 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c2d0 tx=0x7f85e80a5880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69dc74f869d1a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.671+0000 7f85e7400700 1 -- 172.21.15.50:0/2779454230 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e80aa080 con 0x55588d4f8eb0 2024-05-10T21:18:32.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.671+0000 7f85e7400700 1 -- 172.21.15.50:0/2779454230 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e80a9980 con 0x55588d4f8eb0 2024-05-10T21:18:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.671+0000 7f85e7400700 1 -- 172.21.15.50:0/2779454230 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e80a9be0 con 0x55588d4f8eb0 2024-05-10T21:18:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.671+0000 7f860c59ab80 1 -- 172.21.15.50:0/2779454230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 msgr2=0x55588d682bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.671+0000 7f860c59ab80 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c2d0 tx=0x7f85e80a5880 comp rx=0 tx=0).stop 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 -- 172.21.15.50:0/2779454230 shutdown_connections 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d676980 0x55588d69cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d4f8eb0 0x55588d682bb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 --2- 172.21.15.50:0/2779454230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676570 0x55588d691000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 -- 172.21.15.50:0/2779454230 >> 172.21.15.50:0/2779454230 conn(0x55588d537af0 msgr2=0x55588d67b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 -- 172.21.15.50:0/2779454230 shutdown_connections 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 -- 172.21.15.50:0/2779454230 wait complete. 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.672+0000 7f860c59ab80 1 -- start start 2024-05-10T21:18:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 Processor -- start 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d6864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676980 0x55588d686a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81ab040 con 0x55588d4f8eb0 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81ac470 con 0x55588d676980 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81ab6d0 con 0x55588d676570 2024-05-10T21:18:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.673+0000 7f860c59ab80 10 client.0 _ll_register_callbacks cb 0x55588d52c3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59588/0 (socket says 172.21.15.50:59588) 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 -- 172.21.15.50:0/1543357894 learned_addr learned my addr 172.21.15.50:0/1543357894 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85ee000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d6864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f7400700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676980 0x55588d686a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 -- 172.21.15.50:0/1543357894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 msgr2=0x55588d6864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d6864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 -- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676980 msgr2=0x55588d686a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676980 0x55588d686a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85f6a00700 1 -- 172.21.15.50:0/1543357894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55588d6734b0 con 0x55588d4f8eb0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.674+0000 7f85ee000700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d6864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f85f6a00700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f85e80a9680 tx=0x7f85e80cc410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945fd313ecb8a385 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f85e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:18:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e80add30 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f85e80a9be0 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e80a9980 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f85e80b27e0 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f860c59ab80 10 client.16860 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.675+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55588d705040 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e80a9980 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f85e80a8d00 con 0x55588d4f8eb0 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 10 client.16860 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 10 client.16860 resolve_mds: validated gid 24539 aka daemon mds.b 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 20 client.16860 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d705730 0x55588d707b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 4 client.16860 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-10T21:18:32.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55588d5a6470 con 0x55588d705730 2024-05-10T21:18:32.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.676+0000 7f85ee000700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d705730 0x55588d707b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f85ee000700 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d705730 0x55588d707b50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6ccf92f9afae22 server_cookie=bcf0a8cfc3f60b1f in_seq=0 out_seq=0 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f85e4c00700 10 client.16860 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f85e4c00700 1 -- 172.21.15.50:0/1543357894 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55588d5a6470 con 0x55588d705730 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f85e4c00700 10 client.16860 handle_command_reply: tid=0 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f860c59ab80 1 client.16860 shutdown 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 msgr2=0x55588d685f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f85e80a9680 tx=0x7f85e80cc410 comp rx=0 tx=0).stop 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 shutdown_connections 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.677+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55588d705730 0x55588d707b50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55588d676980 0x55588d686a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55588d4f8eb0 0x55588d685f80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 --2- 172.21.15.50:0/1543357894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55588d676570 0x55588d6864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 >> 172.21.15.50:0/1543357894 conn(0x55588d537af0 msgr2=0x55588d681ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f85e4c00700 0 client.16860 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 shutdown_connections 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 1 -- 172.21.15.50:0/1543357894 wait complete. 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.678+0000 7f860c59ab80 20 client.16860 trim_cache size 0 max 16384 2024-05-10T21:18:32.677 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:32.678 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 40, 2024-05-10T21:18:32.678 INFO:teuthology.orchestra.run.smithi050.stdout: "leader": 24539, 2024-05-10T21:18:32.678 INFO:teuthology.orchestra.run.smithi050.stdout: "set_version": 15, 2024-05-10T21:18:32.678 INFO:teuthology.orchestra.run.smithi050.stdout: "sets": {} 2024-05-10T21:18:32.678 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 msgr2=0x7f85dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 0x7f85dc0b4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f85f0011a10 tx=0x7f85f0018000 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 msgr2=0x7f85f819afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85e800e9a0 tx=0x7f85e800ecb0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 shutdown_connections 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85dc0b2880 0x7f85dc0b4d90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85f81045f0 0x7f85f819afb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f8109870 0x7f85f819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 --2- 172.21.15.50:0/2216584294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85f8108f20 0x7f85f819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:32.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 >> 172.21.15.50:0/2216584294 conn(0x7f85f8100040 msgr2=0x7f85f8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:32.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 shutdown_connections 2024-05-10T21:18:32.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:32.679+0000 7f85fda00700 1 -- 172.21.15.50:0/2216584294 wait complete. 2024-05-10T21:18:33.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:33 smithi092 ceph-mon[34415]: pgmap v752: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 521 KiB/s wr, 121 op/s 2024-05-10T21:18:33.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:33 smithi142 ceph-mon[34846]: pgmap v752: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 521 KiB/s wr, 121 op/s 2024-05-10T21:18:33.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:33 smithi050 ceph-mon[29546]: pgmap v752: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 521 KiB/s wr, 121 op/s 2024-05-10T21:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 -- 172.21.15.50:0/2592927654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8108a90 msgr2=0x7fcce810aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 --2- 172.21.15.50:0/2592927654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8108a90 0x7fcce810aeb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009be0 tx=0x7fcce002a6d0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 -- 172.21.15.50:0/2592927654 shutdown_connections 2024-05-10T21:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 --2- 172.21.15.50:0/2592927654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8108a90 0x7fcce810aeb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 --2- 172.21.15.50:0/2592927654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8103d90 0x7fcce81041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 --2- 172.21.15.50:0/2592927654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8102b50 0x7fcce8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 -- 172.21.15.50:0/2592927654 >> 172.21.15.50:0/2592927654 conn(0x7fcce80fdd40 msgr2=0x7fcce81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 -- 172.21.15.50:0/2592927654 shutdown_connections 2024-05-10T21:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.452+0000 7fccece00640 1 -- 172.21.15.50:0/2592927654 wait complete. 2024-05-10T21:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.453+0000 7fccece00640 1 Processor -- start 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 -- start start 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 0x7fcce80fe430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8108a90 0x7fcce80fe970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810e680 con 0x7fcce8102b50 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810e500 con 0x7fcce8103d90 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fccece00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810e800 con 0x7fcce8108a90 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fcce5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fcce5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59612/0 (socket says 172.21.15.50:59612) 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fcce5e00640 1 -- 172.21.15.50:0/2447797179 learned_addr learned my addr 172.21.15.50:0/2447797179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fcce6800640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8108a90 0x7fcce80fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.454+0000 7fcce5400640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 0x7fcce80fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 -- 172.21.15.50:0/2447797179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8108a90 msgr2=0x7fcce80fe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8108a90 0x7fcce80fe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 -- 172.21.15.50:0/2447797179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 msgr2=0x7fcce80fe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 0x7fcce80fe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 -- 172.21.15.50:0/2447797179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce80ff0a0 con 0x7fcce8102b50 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5400640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 0x7fcce80fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fcce5e00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccd800e9f0 tx=0x7fccd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.455+0000 7fccdea00640 1 -- 172.21.15.50:0/2447797179 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd800ccc0 con 0x7fcce8102b50 2024-05-10T21:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.456+0000 7fccdea00640 1 -- 172.21.15.50:0/2447797179 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd80035f0 con 0x7fcce8102b50 2024-05-10T21:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.456+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81a43a0 con 0x7fcce8102b50 2024-05-10T21:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.456+0000 7fccdea00640 1 -- 172.21.15.50:0/2447797179 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd800ce60 con 0x7fcce8102b50 2024-05-10T21:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.456+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a4810 con 0x7fcce8102b50 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.460+0000 7fccf3e7b740 1 Processor -- start 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.460+0000 7fccf3e7b740 1 -- start start 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aeed22e550 con 0x55aeeceff2f0 2024-05-10T21:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aeed1d3bc0 con 0x55aeecf074d0 2024-05-10T21:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aeed22a510 con 0x55aeecf281b0 2024-05-10T21:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fcce6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fcce6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41710/0 (socket says 172.21.15.50:41710) 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fcce6800640 1 -- 172.21.15.50:0/1144107215 learned_addr learned my addr 172.21.15.50:0/1144107215 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fcce5400640 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.461+0000 7fccdea00640 1 -- 172.21.15.50:0/2447797179 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fccd8003aa0 con 0x7fcce8102b50 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 -- 172.21.15.50:0/1144107215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 msgr2=0x55aeed2d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 -- 172.21.15.50:0/1144107215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 msgr2=0x55aeed2ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 -- 172.21.15.50:0/1144107215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeecf40210 con 0x55aeecf281b0 2024-05-10T21:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.462+0000 7fcce6800640 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009be0 tx=0x7fcce0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0d7a4cf57e251b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.463+0000 7fccd7400640 1 -- 172.21.15.50:0/1144107215 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0041030 con 0x55aeecf281b0 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.463+0000 7fccd7400640 1 -- 172.21.15.50:0/1144107215 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce0030530 con 0x55aeecf281b0 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.463+0000 7fccd7400640 1 -- 172.21.15.50:0/1144107215 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0036040 con 0x55aeecf281b0 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.463+0000 7fccdea00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 0x7fcccc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccdea00640 1 -- 172.21.15.50:0/2447797179 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fccd80e2580 con 0x7fcce8102b50 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1144107215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 msgr2=0x55aeed2ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009be0 tx=0x7fcce0005fd0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fcce5e00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 0x7fcccc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1144107215 shutdown_connections 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1144107215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2ca100 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1144107215 >> 172.21.15.50:0/1144107215 conn(0x55aeecf1c300 msgr2=0x55aeed2c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1144107215 shutdown_connections 2024-05-10T21:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.464+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1144107215 wait complete. 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.465+0000 7fcce5e00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 0x7fcccc0b4cf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a9b00 tx=0x7fccd80a92e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.465+0000 7fccf3e7b740 1 -- start start 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.465+0000 7fccf3e7b740 1 Processor -- start 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aeed2d1f70 con 0x55aeeceff2f0 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aeed1e7660 con 0x55aeecf074d0 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aeed2d20f0 con 0x55aeecf281b0 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fccf3e7b740 10 client.0 _ll_register_callbacks cb 0x55aeed1b1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fcce6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fcce5400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fcce6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59624/0 (socket says 172.21.15.50:59624) 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fcce5400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:50978/0 (socket says 172.21.15.50:50978) 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.466+0000 7fcce6800640 1 -- 172.21.15.50:0/3181834868 learned_addr learned my addr 172.21.15.50:0/3181834868 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce6800640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 -- 172.21.15.50:0/3181834868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 msgr2=0x55aeed2db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 -- 172.21.15.50:0/3181834868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 msgr2=0x55aeed2d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 -- 172.21.15.50:0/3181834868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeed00c620 con 0x55aeecf074d0 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcce5400640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcce80ff730 tx=0x7fccd000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41b45f984b3ac8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.467+0000 7fcccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00120d0 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fccd0005020 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce0030530 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fccd0023070 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fccf3e7b740 10 client.27029 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aeed2de000 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.468+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce0030530 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fccd001c470 con 0x55aeecf074d0 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 10 client.27029 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 10 client.27029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 20 client.27029 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecee6d80 0x55aeed2e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 4 client.27029 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T21:18:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeed23c160 con 0x55aeecee6d80 2024-05-10T21:18:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.469+0000 7fcce6800640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecee6d80 0x55aeed2e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.470+0000 7fcce6800640 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecee6d80 0x55aeed2e1830 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7bb5f9f7e1b0c42 server_cookie=97bd298428ef34fe in_seq=0 out_seq=0 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.470+0000 7fcccbe00640 10 client.27029 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fcccbe00640 1 -- 172.21.15.50:0/3181834868 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aeed23c160 con 0x55aeecee6d80 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fcccbe00640 10 client.27029 handle_command_reply: tid=0 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 client.27029 shutdown 2024-05-10T21:18:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 msgr2=0x55aeed2d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcce80ff730 tx=0x7fccd000f790 comp rx=0 tx=0).stop 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 shutdown_connections 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecee6d80 0x55aeed2e1830 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d40a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fcccbe00640 0 client.27029 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3181834868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 >> 172.21.15.50:0/3181834868 conn(0x55aeecf1c300 msgr2=0x55aeed2c4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 shutdown_connections 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.471+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3181834868 wait complete. 2024-05-10T21:18:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.472+0000 7fccf3e7b740 20 client.27029 trim_cache size 0 max 16384 2024-05-10T21:18:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.488+0000 7fccf3e7b740 1 Processor -- start 2024-05-10T21:18:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.488+0000 7fccf3e7b740 1 -- start start 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeeceff2f0 0x55aeed2ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81afcd0 con 0x55aeecf074d0 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ad4e0 con 0x55aeeceff2f0 2024-05-10T21:18:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ab330 con 0x55aeecf281b0 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce6800640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeeceff2f0 0x55aeed2ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59626/0 (socket says 172.21.15.50:59626) 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 -- 172.21.15.50:0/3401849755 learned_addr learned my addr 172.21.15.50:0/3401849755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce6800640 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 -- 172.21.15.50:0/3401849755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 msgr2=0x55aeed2daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 -- 172.21.15.50:0/3401849755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeeceff2f0 msgr2=0x55aeed2ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeeceff2f0 0x55aeed2ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.489+0000 7fcce5400640 1 -- 172.21.15.50:0/3401849755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeed2d3530 con 0x55aeecf074d0 2024-05-10T21:18:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.490+0000 7fcce5400640 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fccd0000c00 tx=0x7fccd0012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744929778d5fe291 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.490+0000 7fccd6a00640 1 -- 172.21.15.50:0/3401849755 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001c470 con 0x55aeecf074d0 2024-05-10T21:18:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.490+0000 7fccd6a00640 1 -- 172.21.15.50:0/3401849755 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd001c610 con 0x55aeecf074d0 2024-05-10T21:18:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.490+0000 7fccd6a00640 1 -- 172.21.15.50:0/3401849755 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00048c0 con 0x55aeecf074d0 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3401849755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 msgr2=0x55aeed2c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fccd0000c00 tx=0x7fccd0012390 comp rx=0 tx=0).stop 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3401849755 shutdown_connections 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed2daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeecf074d0 0x55aeed2c8fa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/3401849755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeeceff2f0 0x55aeed2ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3401849755 >> 172.21.15.50:0/3401849755 conn(0x55aeecf1c300 msgr2=0x55aeed2c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3401849755 shutdown_connections 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.491+0000 7fccf3e7b740 1 -- 172.21.15.50:0/3401849755 wait complete. 2024-05-10T21:18:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 -- start start 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 Processor -- start 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed3503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ab9a0 con 0x55aeeceff2f0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ace10 con 0x55aeecf074d0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.492+0000 7fccf3e7b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ac070 con 0x55aeecf281b0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fccf3e7b740 10 client.0 _ll_register_callbacks cb 0x7fccd0012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59630/0 (socket says 172.21.15.50:59630) 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 -- 172.21.15.50:0/1829205129 learned_addr learned my addr 172.21.15.50:0/1829205129 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce5400640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed3503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 -- 172.21.15.50:0/1829205129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 msgr2=0x55aeed3503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed3503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 -- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 msgr2=0x55aeed2d6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 -- 172.21.15.50:0/1829205129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeed3508e0 con 0x55aeeceff2f0 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce5400640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcce6800640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcce002abf0 tx=0x7fcce0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854204b4cd800de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.493+0000 7fcccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0041040 con 0x55aeeceff2f0 2024-05-10T21:18:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fcce0030a70 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd001c610 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcce0036040 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fccf3e7b740 10 client.16875 fetch_fsmap learned FSMap version 40 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aeed351b30 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.494+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd001c610 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcce0003f00 con 0x55aeeceff2f0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 10 client.16875 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 20 client.16875 populate_metadata read hostname 'smithi050' 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecebc1c0 0x55aeed354150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 4 client.16875 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 --> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aeed235fd0 con 0x55aeecebc1c0 2024-05-10T21:18:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.495+0000 7fcce5400640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecebc1c0 0x55aeed354150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.496+0000 7fcce5400640 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecebc1c0 0x55aeed354150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cc868c83fc30c7 server_cookie=1a1f30c702fa4af8 in_seq=0 out_seq=0 2024-05-10T21:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.496+0000 7fcccbe00640 10 client.16875 ms_handle_connect on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fcccbe00640 1 -- 172.21.15.50:0/1829205129 <== mds.0 v2:172.21.15.92:6838/3200447854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aeed235fd0 con 0x55aeecebc1c0 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fcccbe00640 10 client.16875 handle_command_reply: tid=0 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 client.16875 shutdown 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 msgr2=0x55aeed2d6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcce002abf0 tx=0x7fcce0005fd0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 shutdown_connections 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:6838/3200447854,v1:172.21.15.92:6839/3200447854] conn(0x55aeecebc1c0 0x55aeed354150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aeecf281b0 0x55aeed3503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aeecf074d0 0x55aeed2d6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 --2- 172.21.15.50:0/1829205129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aeeceff2f0 0x55aeed2d6750 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fcccbe00640 0 client.16875 ms_handle_reset on v2:172.21.15.92:6838/3200447854 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 >> 172.21.15.50:0/1829205129 conn(0x55aeecf1c300 msgr2=0x55aeed2c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.503+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 shutdown_connections 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccf3e7b740 1 -- 172.21.15.50:0/1829205129 wait complete. 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccf3e7b740 20 client.16875 trim_cache size 0 max 16384 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-10T21:18:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-10T21:18:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 msgr2=0x7fcccc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 0x7fcccc0b4cf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a9b00 tx=0x7fccd80a92e0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 msgr2=0x7fcce80ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.504+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccd800e9f0 tx=0x7fccd800eeb0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 shutdown_connections 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fcccc0b27e0 0x7fcccc0b4cf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce8108a90 0x7fcce80fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcce8103d90 0x7fcce80fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 --2- 172.21.15.50:0/2447797179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcce8102b50 0x7fcce80ffd90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 >> 172.21.15.50:0/2447797179 conn(0x7fcce80fdd40 msgr2=0x7fcce806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 shutdown_connections 2024-05-10T21:18:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-10T21:18:33.505+0000 7fccece00640 1 -- 172.21.15.50:0/2447797179 wait complete. 2024-05-10T21:18:34.056 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 419, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 378, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error releasing set 'f0125e47': 1 (EPERM) 2024-05-10T21:18:34.057 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-10T21:18:34.068 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:18:34.096 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T21:18:34.107 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T21:18:34.107 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 152, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 419, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/quiescer.py", line 378, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error releasing set 'f0125e47': 1 (EPERM) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b710596db3f44de0a4b72eff459db857656fe05/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-10T21:18:34.108 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T21:18:34.165 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 -- 172.21.15.50:0/123509915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c8106ea0 msgr2=0x7fd7c81072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 --2- 172.21.15.50:0/123509915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c8106ea0 0x7fd7c81072b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009900 tx=0x7fd7b402a400 comp rx=0 tx=0).stop 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 -- 172.21.15.50:0/123509915 shutdown_connections 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 --2- 172.21.15.50:0/123509915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c810bba0 0x7fd7c810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 --2- 172.21.15.50:0/123509915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c8106ea0 0x7fd7c81072b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 --2- 172.21.15.50:0/123509915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8105c60 0x7fd7c8106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 -- 172.21.15.50:0/123509915 >> 172.21.15.50:0/123509915 conn(0x7fd7c8100e50 msgr2=0x7fd7c8103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 -- 172.21.15.50:0/123509915 shutdown_connections 2024-05-10T21:18:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.471+0000 7fd7cea00640 1 -- 172.21.15.50:0/123509915 wait complete. 2024-05-10T21:18:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.472+0000 7fd7cea00640 1 Processor -- start 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 -- start start 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 0x7fd7c819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 0x7fd7c81a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c81117b0 con 0x7fd7c8106ea0 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8111630 con 0x7fd7c810bba0 2024-05-10T21:18:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8111930 con 0x7fd7c8105c60 2024-05-10T21:18:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7ccc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7ccc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51980/0 (socket says 172.21.15.50:51980) 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7ccc00640 1 -- 172.21.15.50:0/777927780 learned_addr learned my addr 172.21.15.50:0/777927780 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7cd600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 0x7fd7c819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.473+0000 7fd7ce000640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 0x7fd7c81a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 -- 172.21.15.50:0/777927780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 msgr2=0x7fd7c819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 0x7fd7c819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 -- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 msgr2=0x7fd7c819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 -- 172.21.15.50:0/777927780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c81a3e90 con 0x7fd7c810bba0 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7cd600640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 0x7fd7c819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ccc00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7ce000640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 0x7fd7c81a3820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000ec50 tx=0x7fd7c000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.474+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0003b70 con 0x7fd7c810bba0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.475+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c0003d10 con 0x7fd7c810bba0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.475+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c81a4180 con 0x7fd7c810bba0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.475+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0010830 con 0x7fd7c810bba0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.475+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c81a46d0 con 0x7fd7c810bba0 2024-05-10T21:18:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.476+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c810a9b0 con 0x7fd7c810bba0 2024-05-10T21:18:34.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.478+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd7c00109d0 con 0x7fd7c810bba0 2024-05-10T21:18:34.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.480+0000 7fd7c6000640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 0x7fd7b00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:34.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.480+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fd7c00e2790 con 0x7fd7c810bba0 2024-05-10T21:18:34.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.480+0000 7fd7cd600640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 0x7fd7b00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:34.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.481+0000 7fd7cd600640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 0x7fd7b00b4d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8106860 tx=0x7fd7bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:34.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.482+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7c00aa510 con 0x7fd7c810bba0 2024-05-10T21:18:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.698+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7c80638d0 con 0x7fd7c810bba0 2024-05-10T21:18:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.700+0000 7fd7c6000640 1 -- 172.21.15.50:0/777927780 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22565 (secure 0 0 0) 0x7fd7c00a9e10 con 0x7fd7c810bba0 2024-05-10T21:18:34.696 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:18:34.696 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:17:28.007498+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 msgr2=0x7fd7b00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 0x7fd7b00b4d70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8106860 tx=0x7fd7bc005f90 comp rx=0 tx=0).stop 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 msgr2=0x7fd7c81a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 0x7fd7c81a3820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000ec50 tx=0x7fd7c000c6a0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 shutdown_connections 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd7b00b2860 0x7fd7b00b4d70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7c810bba0 0x7fd7c81a3820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7c8106ea0 0x7fd7c819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 --2- 172.21.15.50:0/777927780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7c8105c60 0x7fd7c819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 >> 172.21.15.50:0/777927780 conn(0x7fd7c8100e50 msgr2=0x7fd7c8107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.703+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 shutdown_connections 2024-05-10T21:18:34.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:34.704+0000 7fd7cea00640 1 -- 172.21.15.50:0/777927780 wait complete. 2024-05-10T21:18:34.727 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T20:58:45.564017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T21:04:18.047634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T21:04:22.866396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T21:18:34.727 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T21:18:35.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.009+0000 7fbe3d600640 1 -- 172.21.15.50:0/2593613039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 msgr2=0x7fbe381050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.009+0000 7fbe3d600640 1 --2- 172.21.15.50:0/2593613039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe381050b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009900 tx=0x7fbe2402a410 comp rx=0 tx=0).stop 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 -- 172.21.15.50:0/2593613039 shutdown_connections 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 --2- 172.21.15.50:0/2593613039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe381099a0 0x7fbe3810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 --2- 172.21.15.50:0/2593613039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe381050b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 --2- 172.21.15.50:0/2593613039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe38103a60 0x7fbe38104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 -- 172.21.15.50:0/2593613039 >> 172.21.15.50:0/2593613039 conn(0x7fbe380fec50 msgr2=0x7fbe38101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 -- 172.21.15.50:0/2593613039 shutdown_connections 2024-05-10T21:18:35.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.010+0000 7fbe3d600640 1 -- 172.21.15.50:0/2593613039 wait complete. 2024-05-10T21:18:35.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.011+0000 7fbe3d600640 1 Processor -- start 2024-05-10T21:18:35.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.011+0000 7fbe3d600640 1 -- start start 2024-05-10T21:18:35.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe3819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 0x7fbe381a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810f5e0 con 0x7fbe381099a0 2024-05-10T21:18:35.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810f460 con 0x7fbe38104ca0 2024-05-10T21:18:35.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810f760 con 0x7fbe38103a60 2024-05-10T21:18:35.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe3cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 0x7fbe381a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36014/0 (socket says 172.21.15.50:36014) 2024-05-10T21:18:35.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe37e00640 1 -- 172.21.15.50:0/1489488269 learned_addr learned my addr 172.21.15.50:0/1489488269 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.012+0000 7fbe37400640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe3819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 -- 172.21.15.50:0/1489488269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 msgr2=0x7fbe3819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe3819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 -- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 msgr2=0x7fbe381a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 0x7fbe381a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 -- 172.21.15.50:0/1489488269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe381a3ef0 con 0x7fbe38103a60 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe3cc00640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 0x7fbe381a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe37e00640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb90 tx=0x7fbe2800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.013+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003b70 con 0x7fbe38103a60 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.014+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe381a41e0 con 0x7fbe38103a60 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.014+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe2800ee90 con 0x7fbe38103a60 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.014+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280197b0 con 0x7fbe38103a60 2024-05-10T21:18:35.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.014+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a4750 con 0x7fbe38103a60 2024-05-10T21:18:35.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.015+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe381a4a10 con 0x7fbe38103a60 2024-05-10T21:18:35.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.018+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbe28004030 con 0x7fbe38103a60 2024-05-10T21:18:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.019+0000 7fbe34c00640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 0x7fbe1c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.020+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fbe280e2b50 con 0x7fbe38103a60 2024-05-10T21:18:35.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.020+0000 7fbe37400640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 0x7fbe1c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.021+0000 7fbe37400640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 0x7fbe1c0b4cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004770 tx=0x7fbe240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:35.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.022+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe280aa8d0 con 0x7fbe38103a60 2024-05-10T21:18:35.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.240+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe381a4da0 con 0x7fbe38103a60 2024-05-10T21:18:35.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.241+0000 7fbe34c00640 1 -- 172.21.15.50:0/1489488269 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fbe280aa1d0 con 0x7fbe38103a60 2024-05-10T21:18:35.237 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 msgr2=0x7fbe1c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 0x7fbe1c0b4cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004770 tx=0x7fbe240314a0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 msgr2=0x7fbe3819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb90 tx=0x7fbe2800c6a0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 shutdown_connections 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fbe1c0b27e0 0x7fbe1c0b4cf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe381099a0 0x7fbe381a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38104ca0 0x7fbe3819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 --2- 172.21.15.50:0/1489488269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe38103a60 0x7fbe3819c3b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 >> 172.21.15.50:0/1489488269 conn(0x7fbe380fec50 msgr2=0x7fbe380ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 shutdown_connections 2024-05-10T21:18:35.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.244+0000 7fbe3d600640 1 -- 172.21.15.50:0/1489488269 wait complete. 2024-05-10T21:18:35.270 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T21:18:35.341 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:35 smithi050 ceph-mon[29546]: pgmap v753: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 55 op/s 2024-05-10T21:18:35.341 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:35 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/777927780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:18:35.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:35 smithi092 ceph-mon[34415]: pgmap v753: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 55 op/s 2024-05-10T21:18:35.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:35 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/777927780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:18:35.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:35 smithi142 ceph-mon[34846]: pgmap v753: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 55 op/s 2024-05-10T21:18:35.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:35 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/777927780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:18:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 -- 172.21.15.50:0/2598381624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8103c40 msgr2=0x7f04a8104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 --2- 172.21.15.50:0/2598381624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8103c40 0x7f04a8104940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0498009900 tx=0x7f049802a3f0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 -- 172.21.15.50:0/2598381624 shutdown_connections 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 --2- 172.21.15.50:0/2598381624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8109b80 0x7f04a810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 --2- 172.21.15.50:0/2598381624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8104e80 0x7f04a8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 --2- 172.21.15.50:0/2598381624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8103c40 0x7f04a8104940 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.566+0000 7f04ada00640 1 -- 172.21.15.50:0/2598381624 >> 172.21.15.50:0/2598381624 conn(0x7f04a80fee70 msgr2=0x7f04a81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.567+0000 7f04ada00640 1 -- 172.21.15.50:0/2598381624 shutdown_connections 2024-05-10T21:18:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.567+0000 7f04ada00640 1 -- 172.21.15.50:0/2598381624 wait complete. 2024-05-10T21:18:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 Processor -- start 2024-05-10T21:18:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 -- start start 2024-05-10T21:18:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8104e80 0x7f04a819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8109b80 0x7f04a81a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810f790 con 0x7f04a8109b80 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810f610 con 0x7f04a8103c40 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.568+0000 7f04ada00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04a810f910 con 0x7f04a8104e80 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49560/0 (socket says 172.21.15.50:49560) 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8109b80 0x7f04a81a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 -- 172.21.15.50:0/1834152584 learned_addr learned my addr 172.21.15.50:0/1834152584 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6000640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8104e80 0x7f04a819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 -- 172.21.15.50:0/1834152584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8104e80 msgr2=0x7f04a819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8104e80 0x7f04a819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 -- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8109b80 msgr2=0x7f04a81a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8109b80 0x7f04a81a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.569+0000 7f04a6a00640 1 -- 172.21.15.50:0/1834152584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a81a4160 con 0x7f04a8103c40 2024-05-10T21:18:35.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f04a6a00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0498031440 tx=0x7f0498003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:35.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049803f070 con 0x7f04a8103c40 2024-05-10T21:18:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a81a43f0 con 0x7f04a8103c40 2024-05-10T21:18:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0498031c60 con 0x7f04a8103c40 2024-05-10T21:18:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498035030 con 0x7f04a8103c40 2024-05-10T21:18:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.570+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a81a48e0 con 0x7f04a8103c40 2024-05-10T21:18:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.571+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a81a4c00 con 0x7f04a8103c40 2024-05-10T21:18:35.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.574+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0498042070 con 0x7f04a8103c40 2024-05-10T21:18:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.576+0000 7f049f400640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 0x7f048c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.576+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f0498102cd0 con 0x7f04a8103c40 2024-05-10T21:18:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.576+0000 7f04a6000640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 0x7f048c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.577+0000 7f04a6000640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 0x7f048c0b4d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f04a819d6c0 tx=0x7f0494005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.578+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04980ca370 con 0x7f04a8103c40 2024-05-10T21:18:35.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.790+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04a81a4dd0 con 0x7f04a8103c40 2024-05-10T21:18:35.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.791+0000 7f049f400640 1 -- 172.21.15.50:0/1834152584 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f04980ce670 con 0x7f04a8103c40 2024-05-10T21:18:35.787 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 msgr2=0x7f048c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 0x7f048c0b4d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f04a819d6c0 tx=0x7f0494005f90 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 msgr2=0x7f04a819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0498031440 tx=0x7f0498003c10 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 shutdown_connections 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f048c0b27f0 0x7f048c0b4d00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8104e80 0x7f04a819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04a8109b80 0x7f04a81a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 --2- 172.21.15.50:0/1834152584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04a8103c40 0x7f04a819c620 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 >> 172.21.15.50:0/1834152584 conn(0x7f04a80fee70 msgr2=0x7f04a8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:35.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 shutdown_connections 2024-05-10T21:18:35.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:35.796+0000 7f04ada00640 1 -- 172.21.15.50:0/1834152584 wait complete. 2024-05-10T21:18:35.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 -- 172.21.15.50:0/4199095771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030106ea0 msgr2=0x7f50301072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 --2- 172.21.15.50:0/4199095771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030106ea0 0x7f50301072b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f501c009900 tx=0x7f501c02a400 comp rx=0 tx=0).stop 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 -- 172.21.15.50:0/4199095771 shutdown_connections 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 --2- 172.21.15.50:0/4199095771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f503010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 --2- 172.21.15.50:0/4199095771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030106ea0 0x7f50301072b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 --2- 172.21.15.50:0/4199095771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030105c60 0x7f5030106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.116+0000 7f5035200640 1 -- 172.21.15.50:0/4199095771 >> 172.21.15.50:0/4199095771 conn(0x7f5030100e50 msgr2=0x7f5030103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:36.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.117+0000 7f5035200640 1 -- 172.21.15.50:0/4199095771 shutdown_connections 2024-05-10T21:18:36.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.117+0000 7f5035200640 1 -- 172.21.15.50:0/4199095771 wait complete. 2024-05-10T21:18:36.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 Processor -- start 2024-05-10T21:18:36.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 -- start start 2024-05-10T21:18:36.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 0x7f503019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030106ea0 0x7f503019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5030111760 con 0x7f5030105c60 2024-05-10T21:18:36.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50301115e0 con 0x7f503010bba0 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.118+0000 7f5035200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50301118e0 con 0x7f5030106ea0 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 0x7f503019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:49580/0 (socket says 172.21.15.50:49580) 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 -- 172.21.15.50:0/1519216897 learned_addr learned my addr 172.21.15.50:0/1519216897 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:36.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502ea00640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030106ea0 0x7f503019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 -- 172.21.15.50:0/1519216897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030106ea0 msgr2=0x7f503019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030106ea0 0x7f503019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 -- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 msgr2=0x7f503019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 0x7f503019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 -- 172.21.15.50:0/1519216897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50301a3e70 con 0x7f503010bba0 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502f400640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 0x7f503019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.119+0000 7f502fe00640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f502800e970 tx=0x7f502800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:36.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.120+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502800ccc0 con 0x7f503010bba0 2024-05-10T21:18:36.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.120+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50301a4160 con 0x7f503010bba0 2024-05-10T21:18:36.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.120+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502800ce60 con 0x7f503010bba0 2024-05-10T21:18:36.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.120+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028010430 con 0x7f503010bba0 2024-05-10T21:18:36.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.120+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50301a4660 con 0x7f503010bba0 2024-05-10T21:18:36.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.121+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503010a9b0 con 0x7f503010bba0 2024-05-10T21:18:36.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.123+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5028004ff0 con 0x7f503010bba0 2024-05-10T21:18:36.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.125+0000 7f5027e00640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 0x7f50140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.125+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f50280e2590 con 0x7f503010bba0 2024-05-10T21:18:36.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.125+0000 7f502f400640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 0x7f50140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.126+0000 7f502f400640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 0x7f50140b4cf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5030106860 tx=0x7f5020009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:36.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.130+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50280aa280 con 0x7f503010bba0 2024-05-10T21:18:36.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:36 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1489488269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:36 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1834152584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.342+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50300638d0 con 0x7f503010bba0 2024-05-10T21:18:36.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.343+0000 7f5027e00640 1 -- 172.21.15.50:0/1519216897 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f50280a9b80 con 0x7f503010bba0 2024-05-10T21:18:36.339 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-05-10T21:18:36.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 msgr2=0x7f50140b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 0x7f50140b4cf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5030106860 tx=0x7f5020009720 comp rx=0 tx=0).stop 2024-05-10T21:18:36.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 msgr2=0x7f50301a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f502800e970 tx=0x7f502800ed70 comp rx=0 tx=0).stop 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 shutdown_connections 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f50140b27e0 0x7f50140b4cf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5030106ea0 0x7f503019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503010bba0 0x7f50301a3800 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 --2- 172.21.15.50:0/1519216897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5030105c60 0x7f503019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 >> 172.21.15.50:0/1519216897 conn(0x7f5030100e50 msgr2=0x7f5030107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.346+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 shutdown_connections 2024-05-10T21:18:36.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.347+0000 7f5035200640 1 -- 172.21.15.50:0/1519216897 wait complete. 2024-05-10T21:18:36.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:36 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1489488269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:36 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1834152584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.371 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T21:18:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:36 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1489488269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:36 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1834152584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T21:18:36.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 -- 172.21.15.50:0/78376007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 msgr2=0x7fd14410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 --2- 172.21.15.50:0/78376007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd14410dfc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009900 tx=0x7fd13c02a3f0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 -- 172.21.15.50:0/78376007 shutdown_connections 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 --2- 172.21.15.50:0/78376007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd14410dfc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 --2- 172.21.15.50:0/78376007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd1441072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 --2- 172.21.15.50:0/78376007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 0x7fd144106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 -- 172.21.15.50:0/78376007 >> 172.21.15.50:0/78376007 conn(0x7fd144100e50 msgr2=0x7fd144103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 -- 172.21.15.50:0/78376007 shutdown_connections 2024-05-10T21:18:36.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.651+0000 7fd143e00640 1 -- 172.21.15.50:0/78376007 wait complete. 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.652+0000 7fd143e00640 1 Processor -- start 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 -- start start 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 0x7fd14419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd14419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1441117e0 con 0x7fd14410bba0 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111660 con 0x7fd144106ea0 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111960 con 0x7fd144105c60 2024-05-10T21:18:36.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52042/0 (socket says 172.21.15.50:52042) 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd143400640 1 -- 172.21.15.50:0/999743747 learned_addr learned my addr 172.21.15.50:0/999743747 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd142a00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 0x7fd14419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.653+0000 7fd13a000640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd14419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 -- 172.21.15.50:0/999743747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 msgr2=0x7fd14419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 0x7fd14419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 -- 172.21.15.50:0/999743747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 msgr2=0x7fd14419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd14419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 -- 172.21.15.50:0/999743747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1441a3e30 con 0x7fd14410bba0 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd13a000640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd14419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:18:36.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.654+0000 7fd143400640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd13c02a910 tx=0x7fd13c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:36.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.655+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c03f070 con 0x7fd14410bba0 2024-05-10T21:18:36.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.655+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd13c031c50 con 0x7fd14410bba0 2024-05-10T21:18:36.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.655+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1441a40c0 con 0x7fd14410bba0 2024-05-10T21:18:36.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.655+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c035030 con 0x7fd14410bba0 2024-05-10T21:18:36.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.655+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441a4600 con 0x7fd14410bba0 2024-05-10T21:18:36.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.656+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14410a9b0 con 0x7fd14410bba0 2024-05-10T21:18:36.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.658+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd13c042070 con 0x7fd14410bba0 2024-05-10T21:18:36.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.660+0000 7fd13be00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 0x7fd1280b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:18:36.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.660+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fd13c103010 con 0x7fd14410bba0 2024-05-10T21:18:36.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.660+0000 7fd142a00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 0x7fd1280b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:18:36.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.661+0000 7fd142a00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 0x7fd1280b4d70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd144106860 tx=0x7fd134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:18:36.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.663+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd13c0ca710 con 0x7fd14410bba0 2024-05-10T21:18:36.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.919+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd14402de00 con 0x7fd14410bba0 2024-05-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.954+0000 7fd13be00640 1 -- 172.21.15.50:0/999743747 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815639 (secure 0 0 0) 0x7fd13c0cea10 con 0x7fd14410bba0 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 18, 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "active_gid": 14152, 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "active_name": "x", 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addrs": { 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6800", 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 628586267 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6801", 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 628586267 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addr": "172.21.15.50:6801/628586267", 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "active_change": "2024-05-10T20:55:55.390513+0000", 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "active_mgr_features": 4540701547738038271, 2024-05-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "standbys": [ 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 24109, 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "z", 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.969 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.973 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.974 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.975 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.976 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.977 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.978 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.979 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:36.980 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.981 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.982 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:36.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.984 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-10T21:18:36.985 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-10T21:18:36.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.987 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.988 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.989 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.991 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:36.992 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.993 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T21:18:36.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-10T21:18:36.995 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T21:18:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-10T21:18:37.004 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.014 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.030 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.031 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-10T21:18:37.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.037 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.038 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.039 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T21:18:37.040 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.041 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.042 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.043 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.044 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-10T21:18:37.045 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-10T21:18:37.046 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-10T21:18:37.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.048 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.049 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.050 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-10T21:18:37.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.052 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.053 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.054 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T21:18:37.055 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.056 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.059 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.060 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.061 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.062 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.063 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.064 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.065 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.066 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.067 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.069 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.070 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.076 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.077 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.078 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.080 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T21:18:37.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T21:18:37.084 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T21:18:37.085 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.086 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.087 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.088 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.089 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.090 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.091 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.092 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.093 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.094 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.095 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.096 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.097 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T21:18:37.098 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T21:18:37.099 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.100 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.101 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.102 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.103 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.104 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.105 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.107 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T21:18:37.108 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T21:18:37.109 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T21:18:37.110 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T21:18:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T21:18:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T21:18:37.113 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T21:18:37.114 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.115 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.116 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.117 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.118 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.119 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.120 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.121 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.122 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.123 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.124 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.125 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.127 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.128 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.130 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.131 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.133 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.134 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:37.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-10T21:18:37.136 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.138 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.139 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.140 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-10T21:18:37.141 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.142 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.143 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.144 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.145 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.146 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T21:18:37.147 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T21:18:37.148 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.150 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.151 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.152 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.153 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.154 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.155 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.157 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.158 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.159 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.160 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-10T21:18:37.161 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.162 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.163 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.164 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.165 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-10T21:18:37.166 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.169 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.170 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.171 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.172 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.173 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.174 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.175 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.176 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.179 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.180 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.181 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.182 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.185 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.186 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.187 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.188 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.189 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.190 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.191 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.192 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.193 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.194 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.195 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.196 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.197 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.198 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.200 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.201 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.202 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.204 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.206 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.207 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.208 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.209 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.210 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.211 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.212 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.214 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.215 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-10T21:18:37.216 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.217 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.218 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.219 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.221 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.222 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.223 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.224 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.226 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.227 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.228 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.229 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.230 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.231 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.232 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.233 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.235 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.236 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.237 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.238 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.240 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.241 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.242 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.243 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.244 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.245 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.246 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.247 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.248 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.249 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.251 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.252 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.253 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.255 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.256 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.257 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.258 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.259 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.260 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.261 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.264 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.265 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-10T21:18:37.267 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.269 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.270 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.271 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.273 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.275 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.276 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.277 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.278 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.279 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.280 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.281 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.282 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.283 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.285 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T21:18:37.286 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.297 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.300 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T21:18:37.301 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.302 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.303 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.305 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:37.306 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.307 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.308 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.310 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.311 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.312 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.313 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.314 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.316 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.317 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.318 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.319 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.322 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.323 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.324 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.325 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.328 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-10T21:18:37.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.330 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.331 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.333 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.334 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.335 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.336 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-10T21:18:37.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.338 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-10T21:18:37.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.340 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.341 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.342 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.343 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.345 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.347 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.348 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.349 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.351 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.354 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.357 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.358 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.360 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.361 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.362 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.363 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.365 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.367 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.370 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.373 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.374 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.375 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.376 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.378 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.380 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.381 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.382 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.383 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.387 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.388 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.391 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.392 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.393 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.394 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.395 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.396 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.397 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.398 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-10T21:18:37.399 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-10T21:18:37.400 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.402 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.404 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.406 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.407 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-10T21:18:37.408 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-10T21:18:37.409 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.410 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.411 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T21:18:37.412 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.413 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.414 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.415 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.416 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.417 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.418 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.420 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.421 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.423 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.424 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.425 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.426 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.428 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-10T21:18:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.431 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.432 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.434 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.435 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.436 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.437 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.438 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.439 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.440 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.442 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 24113, 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "y", 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.443 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.444 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.445 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.446 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.447 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.449 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.450 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.452 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.453 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.454 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.455 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.456 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.458 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.459 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.460 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-10T21:18:37.461 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-10T21:18:37.462 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-10T21:18:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-10T21:18:37.464 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-10T21:18:37.465 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-10T21:18:37.466 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.467 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.468 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.469 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.470 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.471 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.472 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.473 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-10T21:18:37.474 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.475 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.476 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.477 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.479 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:37.480 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.481 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-10T21:18:37.482 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.483 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.484 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.485 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.486 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.487 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.491 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-10T21:18:37.493 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-10T21:18:37.494 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-10T21:18:37.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-10T21:18:37.496 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.498 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T21:18:37.499 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-10T21:18:37.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.505 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.506 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.507 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.508 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.512 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-10T21:18:37.515 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.516 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.517 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.518 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.519 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.520 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.521 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.522 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.524 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.525 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.526 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.527 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.528 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:37.529 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.530 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.531 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.532 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.533 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.534 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.535 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.536 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.537 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.538 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.539 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.540 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.541 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.542 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.543 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.546 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.547 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.548 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.549 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.550 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.551 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.553 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T21:18:37.555 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.556 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.557 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.558 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.559 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.562 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.563 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.564 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.565 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.566 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.567 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.568 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.569 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.570 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.571 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.572 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.573 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T21:18:37.574 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.575 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.577 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.578 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.579 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.580 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.581 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.582 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.583 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.584 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.585 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.586 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.587 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.588 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.589 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.590 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.591 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.592 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.593 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.594 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.597 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.598 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.603 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.604 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.605 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-10T21:18:37.606 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.607 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.608 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.609 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.610 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.611 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.612 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.613 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.614 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T21:18:37.615 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.616 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.617 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.618 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.620 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.621 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.623 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.624 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.625 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.626 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.627 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.628 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.629 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.630 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.631 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-10T21:18:37.632 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-10T21:18:37.633 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.634 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.635 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.636 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.638 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.639 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.640 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.642 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.643 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.644 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:37.645 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-10T21:18:37.646 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.648 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-10T21:18:37.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-10T21:18:37.650 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.651 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-10T21:18:37.652 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.653 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.656 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-10T21:18:37.659 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-10T21:18:37.661 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-10T21:18:37.662 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.663 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.665 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.666 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.667 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.668 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.670 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.671 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.672 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.674 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.675 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.676 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.677 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.679 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.681 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.682 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.683 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.684 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.685 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.686 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.687 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.691 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.692 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.693 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-10T21:18:37.694 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-10T21:18:37.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-10T21:18:37.696 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.697 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.698 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.699 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.700 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.701 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.702 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.704 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.706 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.707 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.708 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.709 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.710 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.711 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.712 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.713 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.716 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.717 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.718 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.719 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.720 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.721 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.723 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.724 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.725 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.726 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.728 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.729 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.730 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.731 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.732 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.733 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.734 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.735 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.736 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.737 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.738 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.739 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.740 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.741 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.743 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.744 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.745 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.746 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T21:18:37.749 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.750 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.751 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.752 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.753 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.754 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.755 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.757 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.758 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.759 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.760 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.761 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-10T21:18:37.762 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.763 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.764 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.765 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.766 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.767 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.768 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.769 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.770 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.771 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.772 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.773 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.774 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.775 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.776 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.777 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.778 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.779 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.780 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.781 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.782 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.783 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.784 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.785 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.786 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.787 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.788 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.790 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.791 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.792 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.793 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.794 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.795 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.796 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.797 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.798 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.800 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.801 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.802 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.804 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.805 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.807 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.808 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.809 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-10T21:18:37.810 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.811 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.812 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.813 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.814 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.815 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.816 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.817 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.819 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.820 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.821 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-10T21:18:37.822 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.825 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.826 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.827 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.828 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-10T21:18:37.829 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.830 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.831 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.832 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.833 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.834 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.835 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.836 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.837 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.838 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.839 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.840 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.841 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.842 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.843 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.844 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.845 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.846 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.847 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.848 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.849 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.850 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.851 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.852 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.853 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.854 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.855 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.856 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.857 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.861 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.862 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.863 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.864 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.865 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.866 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.868 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-10T21:18:37.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.870 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.873 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.874 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.875 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.876 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.877 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.878 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-10T21:18:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-10T21:18:37.880 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-10T21:18:37.881 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.882 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.883 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.884 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.885 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.886 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.887 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.888 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.889 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.890 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.891 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.892 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.893 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.894 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.895 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.896 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.897 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.898 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.899 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.900 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.901 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.903 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.904 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.905 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.906 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.907 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.908 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.909 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.910 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.911 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.912 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.913 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.914 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.915 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.916 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.917 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-10T21:18:37.918 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-10T21:18:37.919 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.920 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.921 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard", 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-05-10T21:18:37.922 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.923 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.924 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.925 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.926 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.927 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.928 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.929 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.930 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.931 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.932 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.933 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-10T21:18:37.934 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-10T21:18:37.935 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-10T21:18:37.936 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.937 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.938 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.939 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.940 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.941 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.942 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.943 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.944 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.945 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-10T21:18:37.946 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.947 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.948 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.949 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.950 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:37.951 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.952 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:37.953 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-10T21:18:37.954 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T21:18:37.955 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.956 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.957 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.958 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.959 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.960 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.961 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.962 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T21:18:37.963 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.964 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-10T21:18:37.966 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.967 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.970 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.971 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.973 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.974 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.975 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.976 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.979 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T21:18:37.980 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-10T21:18:37.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.983 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.984 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.986 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T21:18:37.988 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.990 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.992 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-10T21:18:37.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-10T21:18:37.994 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-10T21:18:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.997 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:37.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.001 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T21:18:38.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.008 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-10T21:18:38.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.011 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.012 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.013 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.014 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-10T21:18:38.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.017 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.018 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T21:18:38.020 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T21:18:38.021 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.024 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.025 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.026 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.027 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.028 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-10T21:18:38.029 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.030 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.032 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.033 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.034 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.035 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T21:18:38.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T21:18:38.037 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T21:18:38.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.039 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.040 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.042 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T21:18:38.043 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.044 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.047 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.048 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.049 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T21:18:38.050 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T21:18:38.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.054 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.055 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.056 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.057 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.058 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.059 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.060 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.061 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.062 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.063 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.064 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.067 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.069 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.070 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.071 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.073 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.074 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T21:18:38.075 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.078 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.079 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.080 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.081 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.082 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-10T21:18:38.083 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:38.084 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.085 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.086 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.087 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.088 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.089 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-10T21:18:38.090 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T21:18:38.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:38.092 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:38.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.094 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:38.095 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-10T21:18:38.096 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:38.097 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.098 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.099 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.100 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.101 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.102 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.103 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.104 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.105 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.107 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.108 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-10T21:18:38.109 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.111 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.112 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.113 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.114 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.115 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.116 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.117 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.118 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.119 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.120 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.121 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.122 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.123 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.124 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.125 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-10T21:18:38.126 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-10T21:18:38.127 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.128 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.129 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.130 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.131 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.132 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.133 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.134 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.135 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.136 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.138 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.139 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.140 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.141 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-10T21:18:38.142 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.145 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.146 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.151 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.154 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.155 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.156 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.157 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.158 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.159 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.160 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.161 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.162 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.163 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.164 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.166 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.167 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.168 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.169 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.170 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.171 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.172 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.173 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.174 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-10T21:18:38.175 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.176 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-10T21:18:38.177 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.194 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.195 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.196 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.197 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:38.198 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.199 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.200 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.201 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.202 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.203 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.204 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-10T21:18:38.205 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.206 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.207 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.208 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.209 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.210 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.211 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.214 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.215 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.217 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.218 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.220 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.221 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.222 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.223 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.224 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.225 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.226 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.228 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.229 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.230 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.231 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.232 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.233 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.234 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.236 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.238 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.240 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.242 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.243 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.244 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.245 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.247 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.248 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.249 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.250 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-10T21:18:38.251 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-10T21:18:38.252 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.253 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-10T21:18:38.254 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-10T21:18:38.256 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.257 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.258 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.259 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.261 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.263 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.265 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.266 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-10T21:18:38.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.268 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.269 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.270 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.272 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.273 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-10T21:18:38.274 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.276 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.278 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.279 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.281 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.282 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.284 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.286 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.287 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.288 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.289 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.290 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.291 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.292 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.293 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-10T21:18:38.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.296 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.297 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.298 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.299 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.300 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.301 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.302 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.303 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-10T21:18:38.304 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.305 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.306 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.307 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.308 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.309 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.310 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.311 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.312 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.313 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-10T21:18:38.314 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.316 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.317 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.318 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.319 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.321 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.322 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.323 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.324 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.325 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.327 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.328 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.330 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.332 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.333 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.334 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.336 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.343 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.344 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.345 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T21:18:38.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.348 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.349 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.350 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.351 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.352 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-10T21:18:38.354 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-10T21:18:38.355 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.356 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.357 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.358 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-10T21:18:38.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.360 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.361 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.362 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.365 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.366 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.367 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.368 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.370 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.372 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.373 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T21:18:38.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.376 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.377 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.379 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.381 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.382 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.383 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.384 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.386 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.387 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.389 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.391 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-10T21:18:38.394 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-10T21:18:38.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-10T21:18:38.396 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.398 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-10T21:18:38.400 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.403 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-10T21:18:38.405 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard": "https://172.21.15.50:8443/" 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "always_on_modules": { 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "octopus": [ 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-10T21:18:38.406 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "pacific": [ 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-10T21:18:38.407 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "quincy": [ 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-10T21:18:38.408 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "reef": [ 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-10T21:18:38.409 INFO:teuthology.orchestra.run.smithi050.stdout: "squid": [ 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "last_failure_osd_epoch": 3, 2024-05-10T21:18:38.410 INFO:teuthology.orchestra.run.smithi050.stdout: "active_clients": [ 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1669819871 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "libcephsqlite", 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-10T21:18:38.411 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1753504381 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1781604349 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.412 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 914903729 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-10T21:18:38.413 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:38.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:37 smithi142 ceph-mon[34846]: pgmap v754: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.5 KiB/s wr, 26 op/s 2024-05-10T21:18:38.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:37 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1519216897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:18:38.414 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:37 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/999743747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T21:18:38.415 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:37 smithi092 ceph-mon[34415]: pgmap v754: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.5 KiB/s wr, 26 op/s 2024-05-10T21:18:38.415 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:37 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1519216897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:18:38.415 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:37 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/999743747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 msgr2=0x7fd1280b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 0x7fd1280b4d70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd144106860 tx=0x7fd134005f90 comp rx=0 tx=0).stop 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 msgr2=0x7fd1441a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd13c02a910 tx=0x7fd13c003b90 comp rx=0 tx=0).stop 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 shutdown_connections 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd1280b2860 0x7fd1280b4d70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:38.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd14410bba0 0x7fd1441a37c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd144106ea0 0x7fd14419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 --2- 172.21.15.50:0/999743747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd144105c60 0x7fd14419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:18:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.963+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 >> 172.21.15.50:0/999743747 conn(0x7fd144100e50 msgr2=0x7fd144107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:18:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.964+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 shutdown_connections 2024-05-10T21:18:38.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:18:36.964+0000 7fd143e00640 1 -- 172.21.15.50:0/999743747 wait complete. 2024-05-10T21:18:38.457 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T21:18:38.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T21:18:38.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:37 smithi050 ceph-mon[29546]: pgmap v754: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.5 KiB/s wr, 26 op/s 2024-05-10T21:18:38.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:37 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1519216897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T21:18:38.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:37 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/999743747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T21:18:38.940 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:39.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:39 smithi092 ceph-mon[34415]: pgmap v755: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-10T21:18:39.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:39 smithi142 ceph-mon[34846]: pgmap v755: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-10T21:18:39.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:39 smithi050 ceph-mon[29546]: pgmap v755: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-05-10T21:18:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:41 smithi050 ceph-mon[29546]: pgmap v756: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T21:18:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:41 smithi092 ceph-mon[34415]: pgmap v756: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T21:18:41.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:41 smithi142 ceph-mon[34846]: pgmap v756: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 53873, 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 48674, 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 19546884, 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 73173739, 2024-05-10T21:18:41.957 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 19, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 7.176928665, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 3.188151834, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 4.875535318, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.472633952, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 48674, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6.984864086, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000143502 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 82548, 2024-05-10T21:18:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.033898963, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000410 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 194663, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 279765, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 71965746, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 208780058, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 13, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 5, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 51.973322582, 2024-05-10T21:18:41.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 35.024620134, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 31.881278704, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.681199157, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 279764, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 32.216191425, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000115154 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 382147, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.188035981, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000492 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 68163824, 2024-05-10T21:18:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 169132608 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 15456, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 15436, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 5558681, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 95842612, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 20, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 7, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 2.842793697, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 1.316858069, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.569090412, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.788058303, 2024-05-10T21:18:41.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 15433, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2.388388154, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000154758 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 12, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000002292, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000191 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 623600, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 324640 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.962 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 15850, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 118.610996085, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.007483343 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-05-10T21:18:41.963 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 13554, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 8.264597056, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000609753 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 135826, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 127385, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 127385, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6123.545286632, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.048071164 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 8441, 2024-05-10T21:18:41.964 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 2, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 500, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 54, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 2, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 32, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 11, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 11, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 10, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 13, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 27885, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 0, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 0, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 5, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 237214, 2024-05-10T21:18:41.965 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 192778, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 8332, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 358, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 101, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 137, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 19759, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 124, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 5940, 2024-05-10T21:18:41.966 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 23628, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 13975, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 325101311, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 23628, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 273, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 273, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 23355, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 134, 2024-05-10T21:18:41.967 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 24875, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 35726, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 39797, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 13955, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 1327, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 27930 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 730, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 823, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 446, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 778, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 2605, 2024-05-10T21:18:41.968 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 8943, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 8943, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 4, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 10196, 2024-05-10T21:18:41.969 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 137, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 57, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 1, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 78200, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 78200, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-05-10T21:18:41.970 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 96, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 96, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 8, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 176701125, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3192, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 347.047518730, 2024-05-10T21:18:41.971 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.108724160 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 37, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 19903, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 19866, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 36, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 1356, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 1320, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 32, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 47905, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 47873, 2024-05-10T21:18:41.972 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 0, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 22449, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 22449, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 184008, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 223632 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 141050, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 7544, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 135826, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 200, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 15906, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16381, 2024-05-10T21:18:41.973 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 626.860638300, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.038267544 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1999, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 303.990906833, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.152071489 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.974 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 80232, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 4293.529463794, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.053513927 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.975 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.976 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.977 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.977 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-05-10T21:18:41.977 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.977 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 125, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2.701870472, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.021614963 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.978 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5226, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 134.523223577, 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.025741144 2024-05-10T21:18:41.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 307, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.800172442, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002606424 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16407, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 736.611851056, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.044896193 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 59, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 8.546772256, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.144860546 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.981 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.982 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.983 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6649, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 15.980387902, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002403427 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-05-10T21:18:41.984 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 1, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 1, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 0, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 1, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 16986, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 16986, 2024-05-10T21:18:41.985 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 620, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:41.986 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:41.987 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 287751, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 97, 2024-05-10T21:18:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 7360, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 157, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-10T21:18:41.989 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 153869, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 689, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 38060, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 184842996, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 33, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 38060, 2024-05-10T21:18:41.990 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 38060, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 130.953765866, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003440719 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 38060, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 39069 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 38060, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 24042, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 14018, 2024-05-10T21:18:41.991 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 10, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 409, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 3601, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 243, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 54, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 9342, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-05-10T21:18:41.992 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 23628, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 20, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-05-10T21:18:41.993 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-05-10T21:18:41.994 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 84, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 4, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 1504, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 7, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 251 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-05-10T21:18:41.995 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 11008 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 18, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 10, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 9283, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 8943, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.996 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 263988, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 64744404, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 263988, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 263988, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 64744404, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.997 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 38060, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 185746463, 2024-05-10T21:18:41.998 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 38060, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 185746463, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:41.999 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 38060, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 38060, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 38060, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 38060, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 8943, 2024-05-10T21:18:42.000 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 903603, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 8943, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 409, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 903603, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:42.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55ff473c45b0": { 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 78200, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 172506821, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 78200, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3192, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 172506821, 2024-05-10T21:18:42.002 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:42.003 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:42.944 INFO:tasks.check_counter:Daemon mds.a mds.exported=137 2024-05-10T21:18:42.944 INFO:tasks.check_counter:Daemon mds.a mds.imported=124 2024-05-10T21:18:42.944 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=730 2024-05-10T21:18:42.944 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=823 2024-05-10T21:18:42.944 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=54 2024-05-10T21:18:42.944 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T21:18:42.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T21:18:43.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:43 smithi050 ceph-mon[29546]: pgmap v757: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:18:43.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:43 smithi092 ceph-mon[34415]: pgmap v757: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:18:43.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:43 smithi142 ceph-mon[34846]: pgmap v757: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-10T21:18:43.417 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:44 smithi050 ceph-mon[29546]: pgmap v758: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-10T21:18:44.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:44 smithi092 ceph-mon[34415]: pgmap v758: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-10T21:18:44.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:44 smithi142 ceph-mon[34846]: pgmap v758: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-10T21:18:45.860 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.003874889, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000685794, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.861 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 186, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 190272, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.035931546, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.026727275, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.004922073, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:45.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 184, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.021007324, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000114170 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 190272 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 249, 2024-05-10T21:18:45.863 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 244, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 524336, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 149808, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.085837824, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.025753881, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.062652802, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.011175826, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 244, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.027114215, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000111123 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:45.864 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 524336, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 149808 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:45.865 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:45.866 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 147920, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 27, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:45.868 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 247, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 486729, 2024-05-10T21:18:45.869 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 247, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 247, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 486729, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:45.870 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:46.848 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-10T21:18:46.848 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-10T21:18:46.848 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-10T21:18:46.848 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-10T21:18:46.848 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-10T21:18:46.848 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T21:18:46.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T21:18:47.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:47 smithi050 ceph-mon[29546]: pgmap v759: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-10T21:18:47.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:47 smithi142 ceph-mon[34846]: pgmap v759: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-10T21:18:47.353 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:47.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:47 smithi092 ceph-mon[34415]: pgmap v759: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-10T21:18:48.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:48 smithi142 ceph-mon[34846]: pgmap v760: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:18:48.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:48 smithi050 ceph-mon[29546]: pgmap v760: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:18:48.864 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:48 smithi092 ceph-mon[34415]: pgmap v760: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-10T21:18:49.958 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.004238200, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000974329, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:49.959 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 182, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 160, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 110624, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 3, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.031514987, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.024587108, 2024-05-10T21:18:49.960 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003153177, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 180, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.020963508, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000116463 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 110624 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.961 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 245, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 240, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 522624, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 147552, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.084537074, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.033660172, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.046178832, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.010438013, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 240, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.023556298, 2024-05-10T21:18:49.962 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000098151 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 522624, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 147552 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.963 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:49.964 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:49.965 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 149998, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 30, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-10T21:18:49.966 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-10T21:18:49.967 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 242, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 485545, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 242, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 242, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 485545, 2024-05-10T21:18:49.968 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:49.969 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:50.800 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-10T21:18:50.800 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-10T21:18:50.800 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-10T21:18:50.801 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-10T21:18:50.801 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-10T21:18:50.801 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T21:18:50.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T21:18:51.272 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:18:51.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:51 smithi092 ceph-mon[34415]: pgmap v761: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:51.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:51 smithi142 ceph-mon[34846]: pgmap v761: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:51.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:51 smithi050 ceph-mon[29546]: pgmap v761: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:53.337 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:53 smithi050 ceph-mon[29546]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:53.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:53 smithi092 ceph-mon[34415]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:53.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:53 smithi142 ceph-mon[34846]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:53.725 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-10T21:18:53.725 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:53.725 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 253, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 248, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 525424, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 152368, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.087444006, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.038078918, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.035597094, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.010811309, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 248, 2024-05-10T21:18:53.726 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.025585428, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000103167 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 525424, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 152368 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 189, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-05-10T21:18:53.727 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 190800, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.036573026, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.026473412, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.005886561, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 187, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.026765442, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000143130 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:53.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 190800 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001434148, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.001269238, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:18:53.729 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-10T21:18:53.730 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:53.731 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:53.732 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 147937, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 28, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:53.733 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:53.734 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 251, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 487249, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 251, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 251, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 487249, 2024-05-10T21:18:53.735 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-10T21:18:53.736 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-10T21:18:54.653 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T21:18:54.653 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T21:18:54.653 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-10T21:18:54.653 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-10T21:18:54.653 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-10T21:18:54.653 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T21:18:54.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T21:18:54.905 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:18:55.274 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:55 smithi092 ceph-mon[34415]: pgmap v763: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:55.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:55 smithi142 ceph-mon[34846]: pgmap v763: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:55.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:55 smithi050 ceph-mon[29546]: pgmap v763: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-10T21:18:56.721 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-10T21:18:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 29122, 2024-05-10T21:18:56.721 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 28829, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 15914662, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 59236923, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 168, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 2.044507835, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.830972332, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 1.161113836, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.248775101, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 28832, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4.126193538, 2024-05-10T21:18:56.722 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000143111 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37002, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006769758, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000182 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 633376, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 170416 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 23264, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 22186, 2024-05-10T21:18:56.723 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 9868490, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 31267069, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 13, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.369909081, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.493924774, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.636073806, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.289260780, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 22187, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 6.573832880, 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000296292 2024-05-10T21:18:56.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20282, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003422205, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000168 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 404848 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 94074, 2024-05-10T21:18:56.725 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 135167, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 37209535, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 99527339, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 5.721636187, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 2.671914292, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 4.477829461, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.331994249, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 135159, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 9.911738315, 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000073333 2024-05-10T21:18:56.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 156813, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026403473, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000168 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 31832080, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 68204160 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-10T21:18:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13767, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 25.303450204, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001837978 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11352, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.376174107, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000121227 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-10T21:18:56.728 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 53351, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 45756, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 45756, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 659.020903957, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.014402939 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 7594, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 56, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 1, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 1750, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 34, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 1, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 24, 2024-05-10T21:18:56.729 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 6, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 8, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 10, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 13, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 10903, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 1, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 1, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 5, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 90358, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 66205, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 7497, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 375, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 296, 2024-05-10T21:18:56.730 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 52, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 111, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 7699, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 83, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 8721, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 12249, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 12249, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-10T21:18:56.731 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 160, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 10, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 150, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 12089, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 108, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 9282, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 17536, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 5, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 17621, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 7980, 2024-05-10T21:18:56.732 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 195, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 14993 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 836, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 2179, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 1209, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 1884, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 1190, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 6207, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 6207, 2024-05-10T21:18:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 4, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 4499, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 10, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 111, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 35, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-10T21:18:56.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 1, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 40727, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 40725, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 40725, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 2, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 2, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 81, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 80, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 80, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 1, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 1, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-10T21:18:56.735 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 1, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 84793672, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 84797078, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1395, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 42.324909077, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.030340436 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 29, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 11092, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 11063, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 28, 2024-05-10T21:18:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 1608, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 1580, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 24, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 26805, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 26781, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 1, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 12834, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 12833, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 177840, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 231824 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 58399, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 3121, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 53351, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 202, 2024-05-10T21:18:56.737 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 8117, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5983, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 68.619957490, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.011469155 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1003, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 17.269328005, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017217674 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 24217, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 413.822725715, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017088108 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.739 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000176826, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000176826 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-10T21:18:56.740 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-10T21:18:56.741 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 56, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 18.974010062, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.338821608 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.742 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2738, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4.866254943, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001777302 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 307, 2024-05-10T21:18:56.743 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.054693987, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003435485 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5575, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 113.010336489, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.020270912 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 65, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 12.382568272, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.190501050 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.745 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.034588988, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005764831 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.746 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.021654362, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003093480 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-05-10T21:18:56.747 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002702139, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000900713 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5795, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 8.961906679, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001546489 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 6636, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 6636, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 638, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:56.749 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:56.750 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:56.751 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 282412, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 91, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-10T21:18:56.752 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7360, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 122653, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 715, 2024-05-10T21:18:56.753 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 32468, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 98806640, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 32, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 32468, 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-10T21:18:56.754 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 32468, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2684.609304186, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.082684775 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 32468, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 53658 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 32468, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 12658, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 19810, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-10T21:18:56.755 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1388, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 8788, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 243, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1628, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 289, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 34, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 6997, 2024-05-10T21:18:56.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 12270, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 17576, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:18:56.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 5, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-10T21:18:56.758 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 84, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 4, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-10T21:18:56.759 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 3618, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 213, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 609 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 4655 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-10T21:18:56.760 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 18, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 10, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 6439, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 6207, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:56.761 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 146456, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 45916732, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 146456, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 146456, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 45916732, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 32468, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 99429432, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 32468, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 99429432, 2024-05-10T21:18:56.763 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 32468, 2024-05-10T21:18:56.764 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 32468, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 32468, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 32468, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:56.765 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 6207, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 627139, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 6207, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 233, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 627139, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.766 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55757ea685b0": { 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 40727, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 80602774, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 40727, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1395, 2024-05-10T21:18:56.767 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 80602774, 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:56.768 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-10T21:18:57.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:57 smithi092 ceph-mon[34415]: pgmap v764: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:57.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:57 smithi142 ceph-mon[34846]: pgmap v764: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:57.404 INFO:tasks.check_counter:Daemon mds.b mds.exported=111 2024-05-10T21:18:57.404 INFO:tasks.check_counter:Daemon mds.b mds.imported=83 2024-05-10T21:18:57.404 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=836 2024-05-10T21:18:57.404 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2179 2024-05-10T21:18:57.404 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=34 2024-05-10T21:18:57.404 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T21:18:57.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T21:18:57.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:57 smithi050 ceph-mon[29546]: pgmap v764: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-10T21:18:57.646 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:18:59.114 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:18:59 smithi092 ceph-mon[34415]: pgmap v765: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:18:59.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:18:59 smithi142 ceph-mon[34846]: pgmap v765: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:18:59.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:18:59 smithi050 ceph-mon[29546]: pgmap v765: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 439, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 417, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 17431586, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 204468, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.066254180, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.018593697, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.035124044, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.011223572, 2024-05-10T21:18:59.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 417, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.019154023, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000045932 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 612928, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 153264 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:18:59.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 849, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1036, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 33769949, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 745591, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.109324176, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.030023722, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.043475388, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.027398697, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1033, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.045506117, 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000044052 2024-05-10T21:18:59.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 476240 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1702, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1704, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 31801514, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 541294, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-10T21:18:59.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.190131541, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.058469458, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.061564551, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.054494819, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1702, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.069905906, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000041072 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4583, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.129307001, 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000028214 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 10556, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 1, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 10555, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-10T21:18:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 10554, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 1, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-10T21:18:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:18:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:18:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-05-10T21:18:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 43520, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 58, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 5, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 85820190, 2024-05-10T21:18:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 85819616, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 43520 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 20, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 10631, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 10611, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 19, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 513, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 494, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 10556, 2024-05-10T21:18:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 28631, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 18075, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 137812, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215440 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-10T21:18:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-10T21:18:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-10T21:18:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 2325, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:18:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:18:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-10T21:18:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 224107, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 74, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7360, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-05-10T21:18:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 7501249, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 42637, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-10T21:18:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.150912299, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001162698 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2711 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 2710, 2024-05-10T21:18:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1636, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 1072, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-10T21:18:59.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-10T21:18:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 84, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-10T21:18:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 4, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 9, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 9, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-10T21:18:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:18:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2986, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 82717265, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2986, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2986, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 82717265, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2710, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 81630137, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2710, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 81630137, 2024-05-10T21:18:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2710, 2024-05-10T21:18:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2710, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2710, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2710, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55a9fa74e8f0": { 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:18:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:18:59.488 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-10T21:19:00.138 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T21:19:00.138 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T21:19:00.138 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-10T21:19:00.138 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-10T21:19:00.138 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-10T21:19:00.139 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T21:19:00.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T21:19:00.377 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:19:01.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:01 smithi092 ceph-mon[34415]: pgmap v766: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:01.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:01 smithi142 ceph-mon[34846]: pgmap v766: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:01.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:01 smithi050 ceph-mon[29546]: pgmap v766: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:02.137 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-10T21:19:02.137 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:02.137 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 21, 2024-05-10T21:19:02.137 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 23, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3999892, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 6694, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.005092291, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000322309, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.003248136, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000429921, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 21, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003515678, 2024-05-10T21:19:02.138 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000167413 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 134, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 132, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 212182, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 48557, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:02.139 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.016013281, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.005299762, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.008452154, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003165825, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 131, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006725614, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000051340 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 178336, 2024-05-10T21:19:02.140 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 17968 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 209, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 228, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 5160212, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 187768, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.018838605, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.004029884, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.008565005, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.005311412, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 227, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.008595908, 2024-05-10T21:19:02.141 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000037867 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 122896 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-10T21:19:02.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 517, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.017024266, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000032928 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-05-10T21:19:02.144 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 1965, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 1367, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 598, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 11, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-05-10T21:19:02.145 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-05-10T21:19:02.146 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-05-10T21:19:02.147 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-10T21:19:02.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-05-10T21:19:02.149 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 5317, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 6, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 84793672, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 84797078, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 84795426, 2024-05-10T21:19:02.150 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 5317 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 14, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 1831, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 1817, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 13, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 127, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 114, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1965, 2024-05-10T21:19:02.151 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 3995, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 2030, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 119320, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215416 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-05-10T21:19:02.152 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.154 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.155 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.156 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.157 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.157 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-10T21:19:02.157 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.157 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.159 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.162 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 1, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-05-10T21:19:02.164 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 2327, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:02.165 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:02.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 204577, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 53, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-10T21:19:02.167 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7360, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 1430472, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 7829, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:02.168 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 321, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 26, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 4, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 321, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 321, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.221993840, 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000691569 2024-05-10T21:19:02.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 321, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 322 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 321, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 321, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 194, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-05-10T21:19:02.170 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 124, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 2, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:19:02.171 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-10T21:19:02.172 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 7, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 7, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-05-10T21:19:02.173 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-10T21:19:02.174 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 360, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 9334954, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 360, 2024-05-10T21:19:02.175 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 360, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9334954, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:02.176 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 321, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 9099154, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 321, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9099154, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:19:02.177 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 321, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 321, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 321, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 321, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:02.178 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:02.179 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x5556cc92c8f0": { 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:02.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:02.181 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-10T21:19:02.876 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-10T21:19:02.876 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-10T21:19:02.876 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-10T21:19:02.876 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-10T21:19:02.876 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-10T21:19:02.876 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T21:19:02.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T21:19:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T21:19:03.114 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:03 smithi092 ceph-mon[34415]: pgmap v767: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:03.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:03 smithi142 ceph-mon[34846]: pgmap v767: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:03.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:03 smithi050 ceph-mon[29546]: pgmap v767: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001089997, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T21:19:04.707 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000180614, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 257, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 252, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 526512, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 154928, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.040651770, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.015353164, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.019164765, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.006078554, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 252, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.013422512, 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000053263 2024-05-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 526512, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 154928 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 192, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 320, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 191328, 2024-05-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.011769182, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.008939807, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000870767, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 190, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.011186816, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000058877 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.711 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 191328 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 147938, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 28, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 255, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 487769, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 255, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 255, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 487769, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-10T21:19:05.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:05 smithi092 ceph-mon[34415]: pgmap v768: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:05.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:05 smithi142 ceph-mon[34846]: pgmap v768: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:05.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:05 smithi050 ceph-mon[29546]: pgmap v768: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:05.447 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-10T21:19:05.447 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-10T21:19:05.447 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-10T21:19:05.447 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-10T21:19:05.447 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-05-10T21:19:05.447 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T21:19:05.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T21:19:05.701 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:19:07.285 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:07 smithi142 ceph-mon[34846]: pgmap v769: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:07.285 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:07 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:19:07.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:07 smithi092 ceph-mon[34415]: pgmap v769: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:07.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:07 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:19:07.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:07 smithi050 ceph-mon[29546]: pgmap v769: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:07.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:07 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2021, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2021, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 59859616, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 641861, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.224318917, 2024-05-10T21:19:07.568 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.064033828, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.078840694, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.064593134, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2020, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.080225642, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000039715 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-10T21:19:07.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 396, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 583, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 54633028, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 691468, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.075385190, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.017069112, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.041679442, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.011690784, 2024-05-10T21:19:07.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 581, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.032213829, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000055445 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 566432 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:07.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 645, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 625, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 59428665, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 271116, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.107344651, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.024531047, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.063885733, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.016349441, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 624, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.026231930, 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000042038 2024-05-10T21:19:07.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 610208, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 153456 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T21:19:07.573 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4730, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.140892056, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000029786 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.574 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-10T21:19:07.575 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 8953, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 8953, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 8953, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:19:07.576 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T21:19:07.577 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-10T21:19:07.578 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T21:19:07.579 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-10T21:19:07.580 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 78200, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 96, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 8, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 176701125, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 176699987, 2024-05-10T21:19:07.581 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 78200 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 13, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 19650, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 19637, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 11, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 860, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 849, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 8953, 2024-05-10T21:19:07.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 43428, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 34475, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 141820, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215440 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-10T21:19:07.583 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T21:19:07.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T21:19:07.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T21:19:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-10T21:19:07.595 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2356, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:07.596 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:07.597 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:07.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 240177, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 79, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7360, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 157, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:07.599 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 6346504, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 35940, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T21:19:07.600 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 2780, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 2780, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2780, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.157942202, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001135950 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2780, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2781 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.601 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 2780, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2780, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1609, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1169, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-10T21:19:07.602 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T21:19:07.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T21:19:07.604 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 84, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 4, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 11, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 11, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T21:19:07.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:07.607 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 3058, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 173628950, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 3058, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3058, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 173628950, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.608 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2780, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 172527809, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2780, 2024-05-10T21:19:07.609 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 172527809, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2780, 2024-05-10T21:19:07.610 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 2780, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2780, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2780, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:07.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x556934424750": { 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:07.612 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:07.613 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T21:19:08.350 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:19:08.351 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:19:08.351 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:08 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:19:08.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:19:08.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:19:08.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:08 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:19:08.377 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-10T21:19:08.377 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-10T21:19:08.377 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-10T21:19:08.377 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-10T21:19:08.377 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-10T21:19:08.378 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T21:19:08.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T21:19:08.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T21:19:08.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T21:19:08.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:08 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' 2024-05-10T21:19:08.631 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:19:09.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:09 smithi050 ceph-mon[29546]: pgmap v770: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:09.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:09 smithi092 ceph-mon[34415]: pgmap v770: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:09.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:09 smithi142 ceph-mon[34846]: pgmap v770: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:10.259 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T21:19:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 0, 2024-05-10T21:19:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2, 2024-05-10T21:19:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 0, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 0, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.000937053, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000128767, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 190, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 320, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 190976, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.011672545, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.008683734, 2024-05-10T21:19:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000952011, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 188, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.012472595, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000066343 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 190976 2024-05-10T21:19:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 254, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 249, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 525696, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 153008, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.040718066, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.015085396, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.018730327, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.006217871, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 249, 2024-05-10T21:19:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.012572419, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000050491 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 525696, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 153008 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-05-10T21:19:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 147898, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 27, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 252, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 487379, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 252, 2024-05-10T21:19:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 252, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 487379, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:10.270 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T21:19:11.145 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-05-10T21:19:11.146 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-05-10T21:19:11.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-05-10T21:19:11.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-05-10T21:19:11.146 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-05-10T21:19:11.146 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T21:19:11.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T21:19:11.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:11 smithi092 ceph-mon[34415]: pgmap v771: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:11.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:11 smithi142 ceph-mon[34846]: pgmap v771: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:11.386 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:19:11.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:11 smithi050 ceph-mon[29546]: pgmap v771: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:13.080 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 27476, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 32936, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 14813778, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 51548382, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 25, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 1.890018146, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.781528832, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 1.090185750, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.201882946, 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:13.081 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 32935, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.838199806, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000025450 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 40856, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.007084648, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000173 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 4576, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 1888 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:13.082 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 94937, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 137493, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 35965975, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 105959796, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 5.655153898, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 2.667425289, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 4.182118594, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.276341211, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 137492, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.876611822, 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000020922 2024-05-10T21:19:13.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 157453, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.026509834, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000168 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 33154496, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 72135424 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 21185, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 22632, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 8180023, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 19906769, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 20, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-05-10T21:19:13.084 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 1.401658668, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.550278818, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.607335972, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.264354929, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22630, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.621138084, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027447 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20330, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003543237, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000174 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.085 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 400800 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 15946, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 21.414155824, 2024-05-10T21:19:13.086 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001342917 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 18712, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.491832507, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000133167 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 53991, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 47384, 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-10T21:19:13.087 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 47384, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 504.184472115, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.010640394 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 6607, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 16, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 2, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 217, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 23, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 43, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 17, 2024-05-10T21:19:13.088 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 14, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 12, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 18, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 866, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 2, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 2, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 5, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 94518, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 69198, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 6606, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 312, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 279, 2024-05-10T21:19:13.089 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 90, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 256, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 131, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 13053, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 16580, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 13975, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 325101311, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 16580, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T21:19:13.090 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 89, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 89, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 16491, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 226, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 11155, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 20483, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 17271, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 5774, 2024-05-10T21:19:13.091 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 231, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 21289 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 852, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 2621, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 1550, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 2218, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1207, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 2, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 10511, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 10509, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-10T21:19:13.092 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 4, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 4323, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 90, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 23, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.093 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 2, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 43520, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 43520, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 58, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 58, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 5, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-10T21:19:13.094 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 85820190, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1181, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 23.130995064, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.019585939 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 44, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 10735, 2024-05-10T21:19:13.095 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 10691, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 39, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1503, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 1464, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 43, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 29027, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 28984, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 2, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 16647, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 16645, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 179772, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223632 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 61890, 2024-05-10T21:19:13.096 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 1776, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 53991, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 49, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 8527, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3106, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 151.790496416, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.048870088 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000873538, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000291179 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-10T21:19:13.097 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27956, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 246.639354122, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008822412 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.098 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000165985, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000165985 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.099 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.100 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.340005010, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.137647353 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.101 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2914, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 10.145913130, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003481782 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-10T21:19:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 418, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.635959490, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001521434 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3110, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 69.591439612, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.022376668 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-10T21:19:13.103 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 68, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 6.545892630, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.096263126 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-10T21:19:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-10T21:19:13.105 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.106 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 9791, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 16.494372182, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001684646 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 12103, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 12103, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 607, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T21:19:13.108 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:13.109 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:13.110 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 304497, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 108, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 88, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 1, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:13.111 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7360, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 157, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 184072, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 1147, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.112 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 29951, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 87762307, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 27, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 29951, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 29951, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 84.377459047, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002817183 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 29951, 2024-05-10T21:19:13.113 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 30572 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 29951, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 16937, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 13014, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 323, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 1504, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 188, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-10T21:19:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 23, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 10844, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 16589, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-10T21:19:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-10T21:19:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 84, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 4, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 848, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 52, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 171 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-05-10T21:19:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 4662 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 18, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 10, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 10813, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 10509, 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-10T21:19:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 143594, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 42125007, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 143594, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 143594, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 42125007, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 29951, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 88823198, 2024-05-10T21:19:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 29951, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 88823198, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-10T21:19:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 29951, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 29951, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 29951, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 29951, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 10509, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1061713, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 10509, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 323, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1061713, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5598941c4410": { 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 43520, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 81625886, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 43520, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1181, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 81625886, 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:13.125 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T21:19:13.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:13 smithi092 ceph-mon[34415]: pgmap v772: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:13.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:13 smithi142 ceph-mon[34846]: pgmap v772: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:13.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:13 smithi050 ceph-mon[29546]: pgmap v772: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-10T21:19:13.994 INFO:tasks.check_counter:Daemon mds.i mds.exported=90 2024-05-10T21:19:13.994 INFO:tasks.check_counter:Daemon mds.i mds.imported=131 2024-05-10T21:19:13.994 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=852 2024-05-10T21:19:13.995 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2621 2024-05-10T21:19:13.995 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=23 2024-05-10T21:19:13.995 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T21:19:13.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T21:19:14.234 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T21:19:15.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:15 smithi092 ceph-mon[34415]: pgmap v773: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:15.373 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:15 smithi142 ceph-mon[34846]: pgmap v773: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:15.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:15 smithi050 ceph-mon[29546]: pgmap v773: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-10T21:19:15.770 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-10T21:19:15.770 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T21:19:15.770 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 4, 2024-05-10T21:19:15.770 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 4, 2024-05-10T21:19:15.770 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 2992, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 448, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.001363310, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000047056, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000529362, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000731842, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000243947 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:15.771 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 195, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 320, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 270800, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.011758829, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.008231774, 2024-05-10T21:19:15.772 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.001390190, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 193, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.015060951, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000078036 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 270800 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T21:19:15.773 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 256, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 253, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 524336, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 156400, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.040672349, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.015418600, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.018388118, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.006316511, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 252, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.013246629, 2024-05-10T21:19:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000052565 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 524336, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 156400 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-10T21:19:15.775 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-10T21:19:15.776 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-10T21:19:15.777 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 147898, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 27, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-10T21:19:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-10T21:19:15.779 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 258, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 488159, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 258, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 258, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 488159, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-10T21:19:15.780 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:15.781 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-10T21:19:15.781 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-10T21:19:16.525 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-10T21:19:16.526 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-10T21:19:16.526 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-10T21:19:16.526 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-10T21:19:16.526 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-05-10T21:19:16.526 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T21:19:16.537 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T21:19:16.537 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:19:16.538 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T21:19:16.601 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:19:16.601 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T21:19:16.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T21:19:17.148 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:17.348 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:17 smithi050 ceph-mon[29546]: pgmap v774: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:17.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:17 smithi092 ceph-mon[34415]: pgmap v774: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:17.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:17 smithi142 ceph-mon[34846]: pgmap v774: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T21:19:19.149 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:19 smithi050 ceph-mon[29546]: pgmap v775: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:19.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:19 smithi092 ceph-mon[34415]: pgmap v775: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:19.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:19 smithi142 ceph-mon[34846]: pgmap v775: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T21:19:19.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 -- 172.21.15.50:0/3128065899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0109870 msgr2=0x7f53d01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 --2- 172.21.15.50:0/3128065899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0109870 0x7f53d01111d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b0d0 tx=0x7f53c800b3e0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 -- 172.21.15.50:0/3128065899 shutdown_connections 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 --2- 172.21.15.50:0/3128065899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d01045f0 0x7f53d0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 --2- 172.21.15.50:0/3128065899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0109870 0x7f53d01111d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 --2- 172.21.15.50:0/3128065899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d0108f20 0x7f53d0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.511+0000 7f53d5800700 1 -- 172.21.15.50:0/3128065899 >> 172.21.15.50:0/3128065899 conn(0x7f53d0100040 msgr2=0x7f53d0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.512+0000 7f53d5800700 1 -- 172.21.15.50:0/3128065899 shutdown_connections 2024-05-10T21:19:19.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.512+0000 7f53d5800700 1 -- 172.21.15.50:0/3128065899 wait complete. 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.513+0000 7f53d5800700 1 Processor -- start 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.513+0000 7f53d5800700 1 -- start start 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d01045f0 0x7f53d019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 0x7f53d019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 0x7f53d01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:19.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53d01144f0 con 0x7f53d0108f20 2024-05-10T21:19:19.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0114370 con 0x7f53d0109870 2024-05-10T21:19:19.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53d5800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0114670 con 0x7f53d01045f0 2024-05-10T21:19:19.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53ce000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 0x7f53d019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53cf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 0x7f53d01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53cea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d01045f0 0x7f53d019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53ce000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 0x7f53d019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52946/0 (socket says 172.21.15.50:52946) 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.514+0000 7f53ce000700 1 -- 172.21.15.50:0/2516752082 learned_addr learned my addr 172.21.15.50:0/2516752082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 -- 172.21.15.50:0/2516752082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d01045f0 msgr2=0x7f53d019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d01045f0 0x7f53d019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 -- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 msgr2=0x7f53d019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 0x7f53d019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 -- 172.21.15.50:0/2516752082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d01a1ed0 con 0x7f53d0109870 2024-05-10T21:19:19.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53cf400700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 0x7f53d01a1830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b770 tx=0x7f53c8007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.515+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c801f070 con 0x7f53d0109870 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.516+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01a2160 con 0x7f53d0109870 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.516+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c8007dc0 con 0x7f53d0109870 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.516+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c801b430 con 0x7f53d0109870 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.516+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a9ae0 con 0x7f53d0109870 2024-05-10T21:19:19.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.517+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d00512c0 con 0x7f53d0109870 2024-05-10T21:19:19.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.519+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f53c8022070 con 0x7f53d0109870 2024-05-10T21:19:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.521+0000 7f53c7400700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 0x7f53b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:19.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.521+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f53c800f020 con 0x7f53d0109870 2024-05-10T21:19:19.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.521+0000 7f53cea00700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 0x7f53b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:19.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.522+0000 7f53cea00700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 0x7f53b40b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009680 tx=0x7f53c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:19.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.523+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53c8029020 con 0x7f53d0109870 2024-05-10T21:19:19.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.777+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53d01a9cb0 con 0x7f53d0109870 2024-05-10T21:19:19.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.778+0000 7f53c7400700 1 -- 172.21.15.50:0/2516752082 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+589 (secure 0 0 0) 0x7f53c80ab360 con 0x7f53d0109870 2024-05-10T21:19:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/3321516664 2024-05-11T20:55:55.390388+0000 2024-05-10T21:19:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/3321516664 2024-05-11T20:55:55.390388+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/484554811 2024-05-11T20:55:19.361123+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/484554811 2024-05-11T20:55:19.361123+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3150174588 2024-05-11T20:55:19.361123+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2515468284 2024-05-11T20:55:19.361123+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2292381409 2024-05-11T20:55:55.390388+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1334201717 2024-05-11T20:55:55.390388+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/356575456 2024-05-11T20:55:19.361123+0000 2024-05-10T21:19:19.775 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3564243559 2024-05-11T20:55:55.390388+0000 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.782+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 msgr2=0x7f53b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.782+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 0x7f53b40b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009680 tx=0x7f53c0009260 comp rx=0 tx=0).stop 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 msgr2=0x7f53d01a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 0x7f53d01a1830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b770 tx=0x7f53c8007980 comp rx=0 tx=0).stop 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 shutdown_connections 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f53b40b2880 0x7f53b40b4d90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d0109870 0x7f53d01a1830 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53d01045f0 0x7f53d019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 --2- 172.21.15.50:0/2516752082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53d0108f20 0x7f53d019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:19.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 >> 172.21.15.50:0/2516752082 conn(0x7f53d0100040 msgr2=0x7f53d0110e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:19.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 shutdown_connections 2024-05-10T21:19:19.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:19.783+0000 7f53d5800700 1 -- 172.21.15.50:0/2516752082 wait complete. 2024-05-10T21:19:19.781 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-05-10T21:19:20.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:20 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2516752082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T21:19:20.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:20 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2516752082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T21:19:20.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:20 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2516752082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T21:19:20.644 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T21:19:20.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:20.645 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T21:19:20.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:19:20.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:20.807 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T21:19:20.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:20.899 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-10T21:19:20.986 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T21:19:20.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:20.987 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T21:19:21.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:21.067 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-05-10T21:19:21.176 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T21:19:21.176 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-10T21:19:21.176 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T21:19:21.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/aac8e0ab02d1f85232f13ac1b7086b0ee7e1ec457b57059c7073c40b710b5dab/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/aac8e0ab02d1f85232f13ac1b7086b0ee7e1ec457b57059c7073c40b710b5dab/diff,workdir=/var/lib/containers/storage/overlay/aac8e0ab02d1f85232f13ac1b7086b0ee7e1ec457b57059c7073c40b710b5dab/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/48e22b04c6d079d6a384a3b79694b7f9f74376f3be4bb581f019e6764b8b2071/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/48e22b04c6d079d6a384a3b79694b7f9f74376f3be4bb581f019e6764b8b2071/diff,workdir=/var/lib/containers/storage/overlay/48e22b04c6d079d6a384a3b79694b7f9f74376f3be4bb581f019e6764b8b2071/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/f95f302471fbfc9293a510d77d7407a1ef4512261959bf7c5ea099b4cfa89bb6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/f95f302471fbfc9293a510d77d7407a1ef4512261959bf7c5ea099b4cfa89bb6/diff,workdir=/var/lib/containers/storage/overlay/f95f302471fbfc9293a510d77d7407a1ef4512261959bf7c5ea099b4cfa89bb6/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b74863762c357502fcd27f78afdde2e93e06d1632318986f6f8089a6e649f2e2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/b74863762c357502fcd27f78afdde2e93e06d1632318986f6f8089a6e649f2e2/diff,workdir=/var/lib/containers/storage/overlay/b74863762c357502fcd27f78afdde2e93e06d1632318986f6f8089a6e649f2e2/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3d1accf370383761d6740b1f3f82e4938aa5506bd5925f608c121d09a5dc608e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/3d1accf370383761d6740b1f3f82e4938aa5506bd5925f608c121d09a5dc608e/diff,workdir=/var/lib/containers/storage/overlay/3d1accf370383761d6740b1f3f82e4938aa5506bd5925f608c121d09a5dc608e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c77e0ac784236ad1cfe62121f0eef5a7206d8ad4c832c052b7da0dcea13ba589/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/c77e0ac784236ad1cfe62121f0eef5a7206d8ad4c832c052b7da0dcea13ba589/diff,workdir=/var/lib/containers/storage/overlay/c77e0ac784236ad1cfe62121f0eef5a7206d8ad4c832c052b7da0dcea13ba589/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c447221b089cc91a4fb062184350a9e8cc1dce32a494bbc0b4f6535130db85c8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/c447221b089cc91a4fb062184350a9e8cc1dce32a494bbc0b4f6535130db85c8/diff,workdir=/var/lib/containers/storage/overlay/c447221b089cc91a4fb062184350a9e8cc1dce32a494bbc0b4f6535130db85c8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f2d5533f85c7fd29fbc5aa220681494b3a072ab5f7ef40a0aa883c114fe85c3b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/f2d5533f85c7fd29fbc5aa220681494b3a072ab5f7ef40a0aa883c114fe85c3b/diff,workdir=/var/lib/containers/storage/overlay/f2d5533f85c7fd29fbc5aa220681494b3a072ab5f7ef40a0aa883c114fe85c3b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5dd0ca8cbea6170d065452e25f1dd5bda71ff32f36f7066571ff8bf45f4942e1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/5dd0ca8cbea6170d065452e25f1dd5bda71ff32f36f7066571ff8bf45f4942e1/diff,workdir=/var/lib/containers/storage/overlay/5dd0ca8cbea6170d065452e25f1dd5bda71ff32f36f7066571ff8bf45f4942e1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8650fa6007a141ecd21cf7df2148ad520ab7c6848d42b822ae228202916fbfc2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZZ7LWZY367W2CDPHXZQBDP3IU7:/var/lib/containers/storage/overlay/l/JWIMING74SQXMLK5WK4NX5ADDV,upperdir=/var/lib/containers/storage/overlay/8650fa6007a141ecd21cf7df2148ad520ab7c6848d42b822ae228202916fbfc2/diff,workdir=/var/lib/containers/storage/overlay/8650fa6007a141ecd21cf7df2148ad520ab7c6848d42b822ae228202916fbfc2/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 2024-05-10T21:19:21.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:21.221 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-10T21:19:21.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T21:19:21.315 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-05-10T21:19:21.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:21 smithi092 ceph-mon[34415]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:21.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:21 smithi142 ceph-mon[34846]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:21.384 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:21 smithi050 ceph-mon[29546]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T21:19:21.403 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-05-10T21:19:21.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:19:21.406 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T21:19:21.419 INFO:tasks.cephadm:Teardown begin 2024-05-10T21:19:21.419 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T21:19:21.419 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:19:21.894 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:22.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:22 smithi092 ceph-mon[34415]: pgmap v777: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:19:22.372 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:22 smithi050 ceph-mon[29546]: pgmap v777: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:19:22.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:22 smithi142 ceph-mon[34846]: pgmap v777: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 -- 172.21.15.50:0/873994612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8075b60 msgr2=0x7f6ca8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 --2- 172.21.15.50:0/873994612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8075b60 0x7f6ca8075f70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009ca0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 -- 172.21.15.50:0/873994612 shutdown_connections 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 --2- 172.21.15.50:0/873994612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8075b60 0x7f6ca8075f70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 --2- 172.21.15.50:0/873994612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca8115370 0x7f6ca8117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.318+0000 7f6cad600700 1 --2- 172.21.15.50:0/873994612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca80764b0 0x7f6ca8114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.319+0000 7f6cad600700 1 -- 172.21.15.50:0/873994612 >> 172.21.15.50:0/873994612 conn(0x7f6ca8100040 msgr2=0x7f6ca8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.319+0000 7f6cad600700 1 -- 172.21.15.50:0/873994612 shutdown_connections 2024-05-10T21:19:24.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.319+0000 7f6cad600700 1 -- 172.21.15.50:0/873994612 wait complete. 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.320+0000 7f6cad600700 1 Processor -- start 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.320+0000 7f6cad600700 1 -- start start 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 0x7f6ca81a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 0x7f6ca81a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca811aad0 con 0x7f6ca80764b0 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca811a950 con 0x7f6ca8115370 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6cad600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca811ac50 con 0x7f6ca8075b60 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6c9e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 0x7f6ca81a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6ca6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6ca7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 0x7f6ca81a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:24.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6ca6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42096/0 (socket says 172.21.15.50:42096) 2024-05-10T21:19:24.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6c9e000700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 0x7f6ca81a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56312/0 (socket says 172.21.15.50:56312) 2024-05-10T21:19:24.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.321+0000 7f6ca6a00700 1 -- 172.21.15.50:0/3380383800 learned_addr learned my addr 172.21.15.50:0/3380383800 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 -- 172.21.15.50:0/3380383800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 msgr2=0x7f6ca81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 -- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 msgr2=0x7f6ca81a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 0x7f6ca81a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 -- 172.21.15.50:0/3380383800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81a6350 con 0x7f6ca8115370 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca6a00700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6ca7400700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 0x7f6ca81a5cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000efc0 tx=0x7f6ca000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.322+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00092e0 con 0x7f6ca8115370 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.323+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca0004bb0 con 0x7f6ca8115370 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.323+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81a65e0 con 0x7f6ca8115370 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.323+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0004020 con 0x7f6ca8115370 2024-05-10T21:19:24.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.323+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81a6ad0 con 0x7f6ca8115370 2024-05-10T21:19:24.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.324+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca80512c0 con 0x7f6ca8115370 2024-05-10T21:19:24.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.326+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6ca001b030 con 0x7f6ca8115370 2024-05-10T21:19:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.328+0000 7f6c9fe00700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 0x7f6c8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:24.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.328+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f6ca00e1e00 con 0x7f6ca8115370 2024-05-10T21:19:24.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.328+0000 7f6ca6a00700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 0x7f6c8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:24.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.329+0000 7f6ca6a00700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 0x7f6c8c0b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c9800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:24.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.331+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ca0022020 con 0x7f6ca8115370 2024-05-10T21:19:24.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.617+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca802e5f0 con 0x7f6ca8115370 2024-05-10T21:19:24.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.619+0000 7f6c9fe00700 1 -- 172.21.15.50:0/3380383800 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7f6ca0010040 con 0x7f6ca8115370 2024-05-10T21:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:17:38.867551+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":206,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26813":{"gid":26813,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1239523786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1239523786},{"type":"v1","addr":"172.21.15.92:6837","nonce":1239523786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 msgr2=0x7f6c8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 0x7f6c8c0b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c9800b260 comp rx=0 tx=0).stop 2024-05-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 msgr2=0x7f6ca81a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 0x7f6ca81a5cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000efc0 tx=0x7f6ca000c3a0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 shutdown_connections 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f6c8c0b2880 0x7f6c8c0b4d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ca8115370 0x7f6ca81a5cb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca8075b60 0x7f6ca81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 --2- 172.21.15.50:0/3380383800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca80764b0 0x7f6ca81a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 >> 172.21.15.50:0/3380383800 conn(0x7f6ca8100040 msgr2=0x7f6ca8100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 shutdown_connections 2024-05-10T21:19:24.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:24.623+0000 7f6cad600700 1 -- 172.21.15.50:0/3380383800 wait complete. 2024-05-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 40 2024-05-10T21:19:25.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:25 smithi092 ceph-mon[34415]: pgmap v778: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T21:19:25.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:25 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3380383800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:25.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:25 smithi142 ceph-mon[34846]: pgmap v778: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T21:19:25.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:25 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3380383800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:25 smithi050 ceph-mon[29546]: pgmap v778: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-10T21:19:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:25 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3380383800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:25.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T21:19:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:27.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:27 smithi092 ceph-mon[34415]: pgmap v779: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:27.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:27 smithi142 ceph-mon[34846]: pgmap v779: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:27.383 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:27 smithi050 ceph-mon[29546]: pgmap v779: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 -- 172.21.15.50:0/886564573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb70110b00 msgr2=0x7feb70112f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 --2- 172.21.15.50:0/886564573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb70110b00 0x7feb70112f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb68009f90 tx=0x7feb6800b330 comp rx=0 tx=0).stop 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 -- 172.21.15.50:0/886564573 shutdown_connections 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 --2- 172.21.15.50:0/886564573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb70110b00 0x7feb70112f20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 --2- 172.21.15.50:0/886564573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 0x7feb70103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 --2- 172.21.15.50:0/886564573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb701038a0 0x7feb701105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 -- 172.21.15.50:0/886564573 >> 172.21.15.50:0/886564573 conn(0x7feb70078ac0 msgr2=0x7feb70078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:28.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 -- 172.21.15.50:0/886564573 shutdown_connections 2024-05-10T21:19:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.369+0000 7feb74e00700 1 -- 172.21.15.50:0/886564573 wait complete. 2024-05-10T21:19:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.370+0000 7feb74e00700 1 Processor -- start 2024-05-10T21:19:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.370+0000 7feb74e00700 1 -- start start 2024-05-10T21:19:28.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 0x7feb70105230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:28.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:28.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb70110b00 0x7feb70105cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:28.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb701164f0 con 0x7feb70110b00 2024-05-10T21:19:28.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feb70116370 con 0x7feb701038a0 2024-05-10T21:19:28.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb74e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb70116670 con 0x7feb70102f50 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb67e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb6f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 0x7feb70105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb67e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56692/0 (socket says 172.21.15.50:56692) 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb67e00700 1 -- 172.21.15.50:0/2780422510 learned_addr learned my addr 172.21.15.50:0/2780422510 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.371+0000 7feb6fe00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb70110b00 0x7feb70105cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 -- 172.21.15.50:0/2780422510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 msgr2=0x7feb70105230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 0x7feb70105230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 -- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb70110b00 msgr2=0x7feb70105cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb70110b00 0x7feb70105cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 -- 172.21.15.50:0/2780422510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701ad420 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb67e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb5800e940 tx=0x7feb5800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58004220 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb580043c0 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.372+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb701ad6b0 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.373+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58003cb0 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.373+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701adc40 con 0x7feb701038a0 2024-05-10T21:19:28.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.374+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7006be40 con 0x7feb701038a0 2024-05-10T21:19:28.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.376+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb5800c7c0 con 0x7feb701038a0 2024-05-10T21:19:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.378+0000 7feb6cc00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 0x7feb540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.378+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7feb580e22d0 con 0x7feb701038a0 2024-05-10T21:19:28.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.378+0000 7feb6f400700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 0x7feb540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:28.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.379+0000 7feb6f400700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 0x7feb540b4e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:28.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.382+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb580a9fc0 con 0x7feb701038a0 2024-05-10T21:19:28.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.670+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb7002e5f0 con 0x7feb701038a0 2024-05-10T21:19:28.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.672+0000 7feb6cc00700 1 -- 172.21.15.50:0/2780422510 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11862 (secure 0 0 0) 0x7feb580b2040 con 0x7feb701038a0 2024-05-10T21:19:28.668 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:19:28.668 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1323665657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1323665657},{"type":"v1","addr":"172.21.15.50:6835","nonce":1323665657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/82705098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":82705098},{"type":"v1","addr":"172.21.15.50:6839","nonce":82705098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2643414360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2643414360},{"type":"v1","addr":"172.21.15.142:6833","nonce":2643414360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/194068448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":194068448},{"type":"v1","addr":"172.21.15.92:6833","nonce":194068448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1369162577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1369162577},{"type":"v1","addr":"172.21.15.50:6837","nonce":1369162577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/86448197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":86448197},{"type":"v1","addr":"172.21.15.142:6835","nonce":86448197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T21:04:26.742871+0000","modified":"2024-05-10T21:17:38.867551+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24520,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/1464775005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1464775005},{"type":"v1","addr":"172.21.15.92:6835","nonce":1464775005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/924361476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":924361476},{"type":"v1","addr":"172.21.15.142:6837","nonce":924361476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6839/3522449943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3522449943},{"type":"v1","addr":"172.21.15.142:6839","nonce":3522449943}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":206,"addr":"172.21.15.50:6841/908891204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":908891204},{"type":"v1","addr":"172.21.15.50:6841","nonce":908891204}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6839/3200447854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":3200447854},{"type":"v1","addr":"172.21.15.92:6839","nonce":3200447854}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26813":{"gid":26813,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6837/1239523786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1239523786},{"type":"v1","addr":"172.21.15.92:6837","nonce":1239523786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24520,24539,24502]},"id":1}]} 2024-05-10T21:19:28.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 msgr2=0x7feb540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:28.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 0x7feb540b4e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb60009310 comp rx=0 tx=0).stop 2024-05-10T21:19:28.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 msgr2=0x7feb70105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:28.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb5800e940 tx=0x7feb5800ec50 comp rx=0 tx=0).stop 2024-05-10T21:19:28.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 shutdown_connections 2024-05-10T21:19:28.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7feb540b2900 0x7feb540b4e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb70110b00 0x7feb70105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb70102f50 0x7feb70105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 --2- 172.21.15.50:0/2780422510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb701038a0 0x7feb70105770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.675+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 >> 172.21.15.50:0/2780422510 conn(0x7feb70078ac0 msgr2=0x7feb70111d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.676+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 shutdown_connections 2024-05-10T21:19:28.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:28.676+0000 7feb74e00700 1 -- 172.21.15.50:0/2780422510 wait complete. 2024-05-10T21:19:28.673 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 40 2024-05-10T21:19:29.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:29 smithi050 ceph-mon[29546]: pgmap v780: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:19:29.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:29 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/2780422510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:29.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:29 smithi092 ceph-mon[34415]: pgmap v780: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:19:29.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:29 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/2780422510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:29 smithi142 ceph-mon[34846]: pgmap v780: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T21:19:29.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:29 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/2780422510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T21:19:29.464 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T21:19:29.924 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:31.217 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:31 smithi050 ceph-mon[29546]: pgmap v781: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:31.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:31 smithi092 ceph-mon[34415]: pgmap v781: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:31.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:31 smithi142 ceph-mon[34846]: pgmap v781: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 -- 172.21.15.50:0/1143193327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106ed0 msgr2=0x7f5188113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 --2- 172.21.15.50:0/1143193327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106ed0 0x7f5188113360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f518000b0a0 tx=0x7f518000b3b0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 -- 172.21.15.50:0/1143193327 shutdown_connections 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 --2- 172.21.15.50:0/1143193327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106ed0 0x7f5188113360 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 --2- 172.21.15.50:0/1143193327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f518810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 --2- 172.21.15.50:0/1143193327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106580 0x7f5188106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.208+0000 7f518ca00700 1 -- 172.21.15.50:0/1143193327 >> 172.21.15.50:0/1143193327 conn(0x7f5188102080 msgr2=0x7f51881044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:32.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.209+0000 7f518ca00700 1 -- 172.21.15.50:0/1143193327 shutdown_connections 2024-05-10T21:19:32.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.209+0000 7f518ca00700 1 -- 172.21.15.50:0/1143193327 wait complete. 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.210+0000 7f518ca00700 1 Processor -- start 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.210+0000 7f518ca00700 1 -- start start 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106ed0 0x7f518819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f51881a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5188116740 con 0x7f5188106ed0 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51881165c0 con 0x7f5188106580 2024-05-10T21:19:32.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f518ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51881168c0 con 0x7f518810c5c0 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f5185e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f5185e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:56704/0 (socket says 172.21.15.50:56704) 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.211+0000 7f5185e00700 1 -- 172.21.15.50:0/1337868906 learned_addr learned my addr 172.21.15.50:0/1337868906 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f517d400700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106ed0 0x7f518819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5186800700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f51881a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 -- 172.21.15.50:0/1337868906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 msgr2=0x7f51881a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f51881a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 -- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106ed0 msgr2=0x7f518819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106ed0 0x7f518819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 -- 172.21.15.50:0/1337868906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51881a1ee0 con 0x7f5188106580 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5186800700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f51881a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.212+0000 7f5185e00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f517800d980 tx=0x7f517800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.213+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178017070 con 0x7f5188106580 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.213+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178010460 con 0x7f5188106580 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.213+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51881a21d0 con 0x7f5188106580 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.213+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517800de60 con 0x7f5188106580 2024-05-10T21:19:32.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.213+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a9af0 con 0x7f5188106580 2024-05-10T21:19:32.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.214+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880512c0 con 0x7f5188106580 2024-05-10T21:19:32.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.216+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5178010610 con 0x7f5188106580 2024-05-10T21:19:32.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.218+0000 7f517f400700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 0x7f516c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.218+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7f51780e1da0 con 0x7f5188106580 2024-05-10T21:19:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.218+0000 7f517d400700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 0x7f516c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:32.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.219+0000 7f517d400700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 0x7f516c0b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5188103b90 tx=0x7f5170009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:32.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.223+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51780a9a90 con 0x7f5188106580 2024-05-10T21:19:32.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.463+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f518802e5f0 con 0x7f5188106580 2024-05-10T21:19:32.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.465+0000 7f517f400700 1 -- 172.21.15.50:0/1337868906 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22565 (secure 0 0 0) 0x7f51780a9390 con 0x7f5188106580 2024-05-10T21:19:32.461 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-10T21:19:32.461 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":84,"fsid":"55ba4db6-0f0f-11ef-bc98-c7b262605968","created":"2024-05-10T20:54:44.451103+0000","modified":"2024-05-10T21:17:28.007498+0000","last_up_change":"2024-05-10T21:03:05.857252+0000","last_in_change":"2024-05-10T21:02:45.701020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T20:58:45.564017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T21:04:18.047634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T21:04:22.866396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87ed2f28-e9c4-4c8e-a444-06e540010639","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6803","nonce":2398884233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6805","nonce":2398884233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6809","nonce":2398884233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2398884233},{"type":"v1","addr":"172.21.15.50:6807","nonce":2398884233}]},"public_addr":"172.21.15.50:6803/2398884233","cluster_addr":"172.21.15.50:6805/2398884233","heartbeat_back_addr":"172.21.15.50:6809/2398884233","heartbeat_front_addr":"172.21.15.50:6807/2398884233","state":["exists","up"]},{"osd":1,"uuid":"afef3a38-83d4-4d62-9db8-065d439b4fda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6801","nonce":1268741901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6803","nonce":1268741901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6807","nonce":1268741901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1268741901},{"type":"v1","addr":"172.21.15.92:6805","nonce":1268741901}]},"public_addr":"172.21.15.92:6801/1268741901","cluster_addr":"172.21.15.92:6803/1268741901","heartbeat_back_addr":"172.21.15.92:6807/1268741901","heartbeat_front_addr":"172.21.15.92:6805/1268741901","state":["exists","up"]},{"osd":2,"uuid":"b00cad0c-573c-4648-9dee-c63912cde7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6801","nonce":1105819435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6803","nonce":1105819435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6807","nonce":1105819435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1105819435},{"type":"v1","addr":"172.21.15.142:6805","nonce":1105819435}]},"public_addr":"172.21.15.142:6801/1105819435","cluster_addr":"172.21.15.142:6803/1105819435","heartbeat_back_addr":"172.21.15.142:6807/1105819435","heartbeat_front_addr":"172.21.15.142:6805/1105819435","state":["exists","up"]},{"osd":3,"uuid":"0d3deb55-a3e3-4b6d-9a12-6a411e9004ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6811","nonce":1367886270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6813","nonce":1367886270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6817","nonce":1367886270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1367886270},{"type":"v1","addr":"172.21.15.50:6815","nonce":1367886270}]},"public_addr":"172.21.15.50:6811/1367886270","cluster_addr":"172.21.15.50:6813/1367886270","heartbeat_back_addr":"172.21.15.50:6817/1367886270","heartbeat_front_addr":"172.21.15.50:6815/1367886270","state":["exists","up"]},{"osd":4,"uuid":"352735a0-8b3b-47b4-accb-d76af31a6279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6809","nonce":1024551638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6811","nonce":1024551638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6815","nonce":1024551638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1024551638},{"type":"v1","addr":"172.21.15.92:6813","nonce":1024551638}]},"public_addr":"172.21.15.92:6809/1024551638","cluster_addr":"172.21.15.92:6811/1024551638","heartbeat_back_addr":"172.21.15.92:6815/1024551638","heartbeat_front_addr":"172.21.15.92:6813/1024551638","state":["exists","up"]},{"osd":5,"uuid":"23efb5de-d1d9-4654-ab57-81f4a6187db5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6809","nonce":3719494232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6811","nonce":3719494232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6815","nonce":3719494232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3719494232},{"type":"v1","addr":"172.21.15.142:6813","nonce":3719494232}]},"public_addr":"172.21.15.142:6809/3719494232","cluster_addr":"172.21.15.142:6811/3719494232","heartbeat_back_addr":"172.21.15.142:6815/3719494232","heartbeat_front_addr":"172.21.15.142:6813/3719494232","state":["exists","up"]},{"osd":6,"uuid":"fc38b4c9-e54b-4f9c-ba67-a2a431b72cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6819","nonce":2895143943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6821","nonce":2895143943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6825","nonce":2895143943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2895143943},{"type":"v1","addr":"172.21.15.50:6823","nonce":2895143943}]},"public_addr":"172.21.15.50:6819/2895143943","cluster_addr":"172.21.15.50:6821/2895143943","heartbeat_back_addr":"172.21.15.50:6825/2895143943","heartbeat_front_addr":"172.21.15.50:6823/2895143943","state":["exists","up"]},{"osd":7,"uuid":"93d87384-c4c0-4d95-b34e-bd0ff0680647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6817","nonce":477438803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6819","nonce":477438803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6823","nonce":477438803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":477438803},{"type":"v1","addr":"172.21.15.92:6821","nonce":477438803}]},"public_addr":"172.21.15.92:6817/477438803","cluster_addr":"172.21.15.92:6819/477438803","heartbeat_back_addr":"172.21.15.92:6823/477438803","heartbeat_front_addr":"172.21.15.92:6821/477438803","state":["exists","up"]},{"osd":8,"uuid":"b945a33a-16a3-4281-a58f-7a066f04c7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6817","nonce":2574916592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6819","nonce":2574916592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6823","nonce":2574916592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2574916592},{"type":"v1","addr":"172.21.15.142:6821","nonce":2574916592}]},"public_addr":"172.21.15.142:6817/2574916592","cluster_addr":"172.21.15.142:6819/2574916592","heartbeat_back_addr":"172.21.15.142:6823/2574916592","heartbeat_front_addr":"172.21.15.142:6821/2574916592","state":["exists","up"]},{"osd":9,"uuid":"462243ae-aa14-4706-828a-a0d7247e5afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6827","nonce":2893617399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6829","nonce":2893617399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6833","nonce":2893617399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2893617399},{"type":"v1","addr":"172.21.15.50:6831","nonce":2893617399}]},"public_addr":"172.21.15.50:6827/2893617399","cluster_addr":"172.21.15.50:6829/2893617399","heartbeat_back_addr":"172.21.15.50:6833/2893617399","heartbeat_front_addr":"172.21.15.50:6831/2893617399","state":["exists","up"]},{"osd":10,"uuid":"08409a23-50ab-4268-9ad6-c50879b535f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6825","nonce":955638920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6827","nonce":955638920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6831","nonce":955638920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":955638920},{"type":"v1","addr":"172.21.15.92:6829","nonce":955638920}]},"public_addr":"172.21.15.92:6825/955638920","cluster_addr":"172.21.15.92:6827/955638920","heartbeat_back_addr":"172.21.15.92:6831/955638920","heartbeat_front_addr":"172.21.15.92:6829/955638920","state":["exists","up"]},{"osd":11,"uuid":"c6ba8a0f-61ce-48d4-a456-18eca77cbdcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6825","nonce":256997334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6827","nonce":256997334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6831","nonce":256997334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":256997334},{"type":"v1","addr":"172.21.15.142:6829","nonce":256997334}]},"public_addr":"172.21.15.142:6825/256997334","cluster_addr":"172.21.15.142:6827/256997334","heartbeat_back_addr":"172.21.15.142:6831/256997334","heartbeat_front_addr":"172.21.15.142:6829/256997334","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:57:51.774368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:18.226244+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:58:42.801931+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:18.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T20:59:44.919393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:09.040322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:00:44.314272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:11.115505+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:01:35.781319+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:11.053521+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:02:38.559659+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T21:03:03.471231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6800/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/3321516664":"2024-05-11T20:55:55.390388+0000","172.21.15.50:6801/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:6800/484554811":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3150174588":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2515468284":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/2292381409":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/1334201717":"2024-05-11T20:55:55.390388+0000","172.21.15.50:0/356575456":"2024-05-11T20:55:19.361123+0000","172.21.15.50:0/3564243559":"2024-05-11T20:55:55.390388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.468+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 msgr2=0x7f516c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.468+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 0x7f516c0b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5188103b90 tx=0x7f5170009260 comp rx=0 tx=0).stop 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.468+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 msgr2=0x7f518819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.468+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f517800d980 tx=0x7f517800dd40 comp rx=0 tx=0).stop 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.468+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 shutdown_connections 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f516c0b2900 0x7f516c0b4e10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5188106580 0x7f518819d010 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518810c5c0 0x7f51881a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 --2- 172.21.15.50:0/1337868906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5188106ed0 0x7f518819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 >> 172.21.15.50:0/1337868906 conn(0x7f5188102080 msgr2=0x7f51881045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 shutdown_connections 2024-05-10T21:19:32.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:32.469+0000 7f518ca00700 1 -- 172.21.15.50:0/1337868906 wait complete. 2024-05-10T21:19:33.141 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:33 smithi050 ceph-mon[29546]: pgmap v782: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:19:33.141 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:33 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1337868906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:19:33.223 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-10T21:19:33.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:33 smithi092 ceph-mon[34415]: pgmap v782: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:19:33.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:33 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1337868906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:19:33.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:33 smithi142 ceph-mon[34846]: pgmap v782: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T21:19:33.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:33 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1337868906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T21:19:33.715 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:35.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:35 smithi050 ceph-mon[29546]: pgmap v783: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:19:35.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:35 smithi092 ceph-mon[34415]: pgmap v783: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:19:35.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:35 smithi142 ceph-mon[34846]: pgmap v783: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-10T21:19:35.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 -- 172.21.15.50:0/2353258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 msgr2=0x7fe724110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 --2- 172.21.15.50:0/2353258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe724110dc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b0a0 tx=0x7fe71c00b3b0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 -- 172.21.15.50:0/2353258855 shutdown_connections 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 --2- 172.21.15.50:0/2353258855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe724110dc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 --2- 172.21.15.50:0/2353258855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105520 0x7fe724105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 --2- 172.21.15.50:0/2353258855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105e70 0x7fe72410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 -- 172.21.15.50:0/2353258855 >> 172.21.15.50:0/2353258855 conn(0x7fe7240fedc0 msgr2=0x7fe7241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 -- 172.21.15.50:0/2353258855 shutdown_connections 2024-05-10T21:19:35.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.876+0000 7fe729600700 1 -- 172.21.15.50:0/2353258855 wait complete. 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.878+0000 7fe729600700 1 Processor -- start 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.878+0000 7fe729600700 1 -- start start 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.878+0000 7fe729600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.878+0000 7fe729600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105e70 0x7fe72419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.878+0000 7fe729600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe7241a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe729600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe724114490 con 0x7fe72410e900 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe729600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe724114310 con 0x7fe724105520 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe729600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe724114610 con 0x7fe724105e70 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe723400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe7241a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe723400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe7241a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58756/0 (socket says 172.21.15.50:58756) 2024-05-10T21:19:35.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe723400700 1 -- 172.21.15.50:0/1732929214 learned_addr learned my addr 172.21.15.50:0/1732929214 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:44278/0 (socket says 172.21.15.50:44278) 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722000700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105e70 0x7fe72419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 -- 172.21.15.50:0/1732929214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105e70 msgr2=0x7fe72419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105e70 0x7fe72419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 -- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 msgr2=0x7fe7241a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe7241a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:35.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.879+0000 7fe722a00700 1 -- 172.21.15.50:0/1732929214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a1f10 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe722a00700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9f0 tx=0x7fe71400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cbd0 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a2200 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a9b20 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71400cd70 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.880+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010e60 con 0x7fe724105520 2024-05-10T21:19:35.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.882+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704005330 con 0x7fe724105520 2024-05-10T21:19:35.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.884+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe714010430 con 0x7fe724105520 2024-05-10T21:19:35.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.885+0000 7fe71b400700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 0x7fe7080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:35.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.886+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8177+0+0 (secure 0 0 0) 0x7fe7140e2420 con 0x7fe724105520 2024-05-10T21:19:35.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.886+0000 7fe722000700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 0x7fe7080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:35.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.887+0000 7fe722000700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 0x7fe7080b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe724110670 tx=0x7fe710009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:35.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:35.890+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7140aa110 con 0x7fe724105520 2024-05-10T21:19:36.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.190+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe704005620 con 0x7fe724105520 2024-05-10T21:19:36.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.204+0000 7fe71b400700 1 -- 172.21.15.50:0/1732929214 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7fe7140a9a10 con 0x7fe724105520 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 msgr2=0x7fe7080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 0x7fe7080b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe724110670 tx=0x7fe710009260 comp rx=0 tx=0).stop 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 msgr2=0x7fe72419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9f0 tx=0x7fe71400ed00 comp rx=0 tx=0).stop 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 shutdown_connections 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fe7080b2880 0x7fe7080b4d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe724105520 0x7fe72419d040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72410e900 0x7fe7241a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:36.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 --2- 172.21.15.50:0/1732929214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe724105e70 0x7fe72419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 >> 172.21.15.50:0/1732929214 conn(0x7fe7240fedc0 msgr2=0x7fe7241031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 shutdown_connections 2024-05-10T21:19:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:36.208+0000 7fe729600700 1 -- 172.21.15.50:0/1732929214 wait complete. 2024-05-10T21:19:36.205 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T21:19:36.593 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29542]: 2024-05-10T21:19:36.192+0000 7f28a3a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T21:19:36.593 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:19:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.593 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:19:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.593 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:19:36 smithi050 ceph-mds[69436]: -- 172.21.15.50:0/1589123155 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d49703a40 con 0x556d4974b600 2024-05-10T21:19:36.593 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:19:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: starting mds.a at 2024-05-10T21:19:36.614 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.614 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.614 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:19:36 smithi092 ceph-mds[64813]: -- 172.21.15.92:0/2674196256 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651f0719a40 con 0x5651f0761600 2024-05-10T21:19:36.614 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: starting mds.b at 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:19:36 smithi092 ceph-mds[64505]: -- 172.21.15.92:0/2220417832 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608a1be7a40 con 0x5608a1c2f600 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: starting mds.h at 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e[64183]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e[64183]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:19:36 smithi092 ceph-mds[64187]: -- 172.21.15.92:0/2221158157 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e43d61a40 con 0x557e43da9600 2024-05-10T21:19:36.615 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:19:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e[64183]: starting mds.e at 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:19:36 smithi142 ceph-mds[65589]: -- 172.21.15.142:0/1056275007 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555988bb3a40 con 0x555988bfba80 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: starting mds.c at 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: ignoring --setuser ceph since I am not root 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: ignoring --setgroup ceph since I am not root 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:19:36 smithi142 ceph-mds[65896]: -- 172.21.15.142:0/278590178 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56227cc49a40 con 0x56227cc91180 2024-05-10T21:19:36.625 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:19:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: starting mds.i at 2024-05-10T21:19:37.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T21:19:37.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: pgmap v784: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:19:37.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1732929214' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T21:19:37.365 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T21:19:37.366 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:37 smithi092 ceph-mon[34415]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T21:19:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: pgmap v784: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:19:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1732929214' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T21:19:37.376 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:37 smithi142 ceph-mon[34846]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T21:19:37.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: pgmap v784: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1732929214' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-05-10T21:19:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T21:19:37.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T21:19:37.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:37 smithi050 ceph-mon[29546]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T21:19:37.506 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6834/665844433,v1:172.21.15.92:6835/665844433] up:boot 2024-05-10T21:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6836/3654069533,v1:172.21.15.142:6838/3654069533] up:boot 2024-05-10T21:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.142:6837/3595810458,v1:172.21.15.142:6839/3595810458] up:boot 2024-05-10T21:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6836/3817114935,v1:172.21.15.92:6838/3817114935] up:boot 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.92:6837/2922329239,v1:172.21.15.92:6839/2922329239] up:boot 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: mds.? [v2:172.21.15.50:6840/64751115,v1:172.21.15.50:6841/64751115] up:boot 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:38 smithi050 ceph-mon[29546]: pgmap v786: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:38.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6834/665844433,v1:172.21.15.92:6835/665844433] up:boot 2024-05-10T21:19:38.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6836/3654069533,v1:172.21.15.142:6838/3654069533] up:boot 2024-05-10T21:19:38.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.142:6837/3595810458,v1:172.21.15.142:6839/3595810458] up:boot 2024-05-10T21:19:38.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6836/3817114935,v1:172.21.15.92:6838/3817114935] up:boot 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.92:6837/2922329239,v1:172.21.15.92:6839/2922329239] up:boot 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: mds.? [v2:172.21.15.50:6840/64751115,v1:172.21.15.50:6841/64751115] up:boot 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:19:38.615 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:38 smithi092 ceph-mon[34415]: pgmap v786: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6834/665844433,v1:172.21.15.92:6835/665844433] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6836/3654069533,v1:172.21.15.142:6838/3654069533] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.142:6837/3595810458,v1:172.21.15.142:6839/3595810458] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6836/3817114935,v1:172.21.15.92:6838/3817114935] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.92:6837/2922329239,v1:172.21.15.92:6839/2922329239] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: mds.? [v2:172.21.15.50:6840/64751115,v1:172.21.15.50:6841/64751115] up:boot 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T21:19:38.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T21:19:38.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T21:19:38.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T21:19:38.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T21:19:38.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: from='mgr.14152 172.21.15.50:0/3856274104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T21:19:38.626 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:38 smithi142 ceph-mon[34846]: pgmap v786: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 -- 172.21.15.50:0/1818246972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a8109850 msgr2=0x7ff2a81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1818246972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a8109850 0x7ff2a81111b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009fc0 tx=0x7ff2a000b330 comp rx=0 tx=0).stop 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 -- 172.21.15.50:0/1818246972 shutdown_connections 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1818246972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a8109850 0x7ff2a81111b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1818246972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a81045d0 0x7ff2a81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1818246972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 0x7ff2a8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:39.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 -- 172.21.15.50:0/1818246972 >> 172.21.15.50:0/1818246972 conn(0x7ff2a80ffea0 msgr2=0x7ff2a81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:39.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 -- 172.21.15.50:0/1818246972 shutdown_connections 2024-05-10T21:19:39.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.809+0000 7ff2ad600700 1 -- 172.21.15.50:0/1818246972 wait complete. 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.810+0000 7ff2ad600700 1 Processor -- start 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 -- start start 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 0x7ff2a8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a81144d0 con 0x7ff2a8109850 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8114350 con 0x7ff2a8108f00 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.811+0000 7ff2ad600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a8114650 con 0x7ff2a81045d0 2024-05-10T21:19:39.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:39.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:39.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff29fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 0x7ff2a8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:39.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49920/0 (socket says 172.21.15.50:49920) 2024-05-10T21:19:39.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a7400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58788/0 (socket says 172.21.15.50:58788) 2024-05-10T21:19:39.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 -- 172.21.15.50:0/1662418048 learned_addr learned my addr 172.21.15.50:0/1662418048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 -- 172.21.15.50:0/1662418048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 msgr2=0x7ff2a8077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 0x7ff2a8077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 -- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 msgr2=0x7ff2a8077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 -- 172.21.15.50:0/1662418048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a81a91c0 con 0x7ff2a81045d0 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a7400700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.812+0000 7ff2a6a00700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ec10 tx=0x7ff29800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.813+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29800cc90 con 0x7ff2a81045d0 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.813+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a81a94b0 con 0x7ff2a81045d0 2024-05-10T21:19:39.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.813+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29800ce30 con 0x7ff2a81045d0 2024-05-10T21:19:39.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.813+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298010430 con 0x7ff2a81045d0 2024-05-10T21:19:39.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.813+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a81a99f0 con 0x7ff2a81045d0 2024-05-10T21:19:39.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.814+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a80512c0 con 0x7ff2a81045d0 2024-05-10T21:19:39.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.817+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff298004000 con 0x7ff2a81045d0 2024-05-10T21:19:39.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.818+0000 7ff29f400700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 0x7ff28c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:39.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.819+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8435+0+0 (secure 0 0 0) 0x7ff2980e26b0 con 0x7ff2a81045d0 2024-05-10T21:19:39.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.819+0000 7ff29fe00700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 0x7ff28c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:39.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.820+0000 7ff29fe00700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 0x7ff28c0b4d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8103520 tx=0x7ff290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:39.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:39.820+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2980aa2a0 con 0x7ff2a81045d0 2024-05-10T21:19:40.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:40.115+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff2a8078100 con 0x7ff2a81045d0 2024-05-10T21:19:41.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.047+0000 7ff29f400700 1 -- 172.21.15.50:0/1662418048 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7ff2980b2320 con 0x7ff2a81045d0 2024-05-10T21:19:41.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.050+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 msgr2=0x7ff28c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:41.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.050+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 0x7ff28c0b4d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8103520 tx=0x7ff290009260 comp rx=0 tx=0).stop 2024-05-10T21:19:41.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 msgr2=0x7ff2a8076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ec10 tx=0x7ff29800ef20 comp rx=0 tx=0).stop 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 shutdown_connections 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7ff28c0b2880 0x7ff28c0b4d90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2a8109850 0x7ff2a8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2a81045d0 0x7ff2a8076bd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:41.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 --2- 172.21.15.50:0/1662418048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a8108f00 0x7ff2a8077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:41.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 >> 172.21.15.50:0/1662418048 conn(0x7ff2a80ffea0 msgr2=0x7ff2a8108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:41.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 shutdown_connections 2024-05-10T21:19:41.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:41.051+0000 7ff2ad600700 1 -- 172.21.15.50:0/1662418048 wait complete. 2024-05-10T21:19:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:41 smithi092 ceph-mon[34415]: pgmap v787: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:41 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1662418048' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:41 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:41 smithi142 ceph-mon[34846]: pgmap v787: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:41.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:41 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1662418048' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:41 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:41 smithi050 ceph-mon[29546]: pgmap v787: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-10T21:19:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:41 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1662418048' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:41 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T21:19:41.867 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T21:19:42.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:42 smithi050 ceph-mon[29546]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T21:19:42.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:42 smithi050 ceph-mon[29546]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T21:19:42.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:42 smithi050 ceph-mon[29546]: Cluster is now healthy 2024-05-10T21:19:42.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:42 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T21:19:42.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:42 smithi050 ceph-mon[29546]: fsmap 12 up:standby 2024-05-10T21:19:42.311 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:42.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:42 smithi092 ceph-mon[34415]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T21:19:42.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:42 smithi092 ceph-mon[34415]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T21:19:42.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:42 smithi092 ceph-mon[34415]: Cluster is now healthy 2024-05-10T21:19:42.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:42 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T21:19:42.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:42 smithi092 ceph-mon[34415]: fsmap 12 up:standby 2024-05-10T21:19:42.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:42 smithi142 ceph-mon[34846]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T21:19:42.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:42 smithi142 ceph-mon[34846]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T21:19:42.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:42 smithi142 ceph-mon[34846]: Cluster is now healthy 2024-05-10T21:19:42.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:42 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T21:19:42.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:42 smithi142 ceph-mon[34846]: fsmap 12 up:standby 2024-05-10T21:19:43.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:43 smithi050 ceph-mon[29546]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-10T21:19:43.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:43 smithi092 ceph-mon[34415]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-10T21:19:43.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:43 smithi142 ceph-mon[34846]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 -- 172.21.15.50:0/217016979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8075d30 msgr2=0x7f85b8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 --2- 172.21.15.50:0/217016979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8075d30 0x7f85b8076140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009990 tx=0x7f85ac009ca0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 -- 172.21.15.50:0/217016979 shutdown_connections 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 --2- 172.21.15.50:0/217016979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8110ba0 0x7f85b8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 --2- 172.21.15.50:0/217016979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8075d30 0x7f85b8076140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 --2- 172.21.15.50:0/217016979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 0x7f85b8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.571+0000 7f85bfc00700 1 -- 172.21.15.50:0/217016979 >> 172.21.15.50:0/217016979 conn(0x7f85b8100040 msgr2=0x7f85b8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.572+0000 7f85bfc00700 1 -- 172.21.15.50:0/217016979 shutdown_connections 2024-05-10T21:19:44.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.572+0000 7f85bfc00700 1 -- 172.21.15.50:0/217016979 wait complete. 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.573+0000 7f85bfc00700 1 Processor -- start 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.573+0000 7f85bfc00700 1 -- start start 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 0x7f85b819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8110ba0 0x7f85b81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8116690 con 0x7f85b8110ba0 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8116510 con 0x7f85b8075d30 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8116810 con 0x7f85b8076680 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bd200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bdc00700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8110ba0 0x7f85b81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bd200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:53342/0 (socket says 172.21.15.50:53342) 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85bd200700 1 -- 172.21.15.50:0/1681553879 learned_addr learned my addr 172.21.15.50:0/1681553879 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.574+0000 7f85b7e00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 0x7f85b819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 -- 172.21.15.50:0/1681553879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 msgr2=0x7f85b819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 0x7f85b819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 -- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8110ba0 msgr2=0x7f85b81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8110ba0 0x7f85b81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 -- 172.21.15.50:0/1681553879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b81a1ef0 con 0x7f85b8075d30 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bd200700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0057f0 tx=0x7f85ac00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac026070 con 0x7f85b8075d30 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b81a2180 con 0x7f85b8075d30 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ac021410 con 0x7f85b8075d30 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.575+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017650 con 0x7f85b8075d30 2024-05-10T21:19:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.576+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a9b00 con 0x7f85b8075d30 2024-05-10T21:19:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.577+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b80512c0 con 0x7f85b8075d30 2024-05-10T21:19:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.579+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f85ac029070 con 0x7f85b8075d30 2024-05-10T21:19:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.581+0000 7f85b5600700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 0x7f85a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.581+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8435+0+0 (secure 0 0 0) 0x7f85ac0e30e0 con 0x7f85b8075d30 2024-05-10T21:19:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.581+0000 7f85b7e00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 0x7f85a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.582+0000 7f85b7e00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 0x7f85a00b4d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85b8103490 tx=0x7f85a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:44.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.586+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85ac013070 con 0x7f85b8075d30 2024-05-10T21:19:44.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:44.822+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85b802e5f0 con 0x7f85b8075d30 2024-05-10T21:19:45.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.078+0000 7f85b5600700 1 -- 172.21.15.50:0/1681553879 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7f85ac0aad60 con 0x7f85b8075d30 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 msgr2=0x7f85a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 0x7f85a00b4d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85b8103490 tx=0x7f85a4009260 comp rx=0 tx=0).stop 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 msgr2=0x7f85b819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0057f0 tx=0x7f85ac00bfd0 comp rx=0 tx=0).stop 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 shutdown_connections 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7f85a00b2880 0x7f85a00b4d90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:45.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b8075d30 0x7f85b819d020 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:45.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85b8110ba0 0x7f85b81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:45.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 --2- 172.21.15.50:0/1681553879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85b8076680 0x7f85b819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:45.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.084+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 >> 172.21.15.50:0/1681553879 conn(0x7f85b8100040 msgr2=0x7f85b81006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:45.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.085+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 shutdown_connections 2024-05-10T21:19:45.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:45.085+0000 7f85bfc00700 1 -- 172.21.15.50:0/1681553879 wait complete. 2024-05-10T21:19:45.082 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-05-10T21:19:45.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:45 smithi092 ceph-mon[34415]: pgmap v789: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-10T21:19:45.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:45 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/1681553879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:45 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:45 smithi142 ceph-mon[34846]: pgmap v789: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-10T21:19:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:45 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/1681553879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:45 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:45 smithi050 ceph-mon[29546]: pgmap v789: 97 pgs: 97 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-10T21:19:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:45 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/1681553879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:45 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:45.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05 shell --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T21:19:46.335 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T21:19:46.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:46 smithi092 ceph-mon[34415]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:46.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:46 smithi092 ceph-mon[34415]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T21:19:46.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:46 smithi142 ceph-mon[34846]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:46.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:46 smithi142 ceph-mon[34846]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T21:19:46.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:46 smithi050 ceph-mon[29546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:46.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:46 smithi050 ceph-mon[29546]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T21:19:47.348 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:47 smithi050 ceph-mon[29546]: pgmap v791: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:47.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:47 smithi092 ceph-mon[34415]: pgmap v791: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:47.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:47 smithi142 ceph-mon[34846]: pgmap v791: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:48.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:48 smithi092 ceph-mon[34415]: pgmap v792: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:48.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:48 smithi142 ceph-mon[34846]: pgmap v792: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:48.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:48 smithi050 ceph-mon[29546]: pgmap v792: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 -- 172.21.15.50:0/1826338939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c109680 msgr2=0x7fd31c110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 --2- 172.21.15.50:0/1826338939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c109680 0x7fd31c110fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd31400b0d0 tx=0x7fd31400b3e0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 -- 172.21.15.50:0/1826338939 shutdown_connections 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 --2- 172.21.15.50:0/1826338939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c109680 0x7fd31c110fe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 --2- 172.21.15.50:0/1826338939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c104400 0x7fd31c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 --2- 172.21.15.50:0/1826338939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c108d30 0x7fd31c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.701+0000 7fd31b200700 1 -- 172.21.15.50:0/1826338939 >> 172.21.15.50:0/1826338939 conn(0x7fd31c079d10 msgr2=0x7fd31c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.702+0000 7fd31b200700 1 -- 172.21.15.50:0/1826338939 shutdown_connections 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.702+0000 7fd31b200700 1 -- 172.21.15.50:0/1826338939 wait complete. 2024-05-10T21:19:48.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.703+0000 7fd31b200700 1 Processor -- start 2024-05-10T21:19:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.703+0000 7fd31b200700 1 -- start start 2024-05-10T21:19:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:48.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 0x7fd31c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c109680 0x7fd31c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1142b0 con 0x7fd31c108d30 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c114130 con 0x7fd31c104400 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c114430 con 0x7fd31c109680 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd319e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd319400700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 0x7fd31c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd319e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.50:53366/0 (socket says 172.21.15.50:53366) 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd319e00700 1 -- 172.21.15.50:0/3998519267 learned_addr learned my addr 172.21.15.50:0/3998519267 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-10T21:19:48.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.704+0000 7fd31a800700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c109680 0x7fd31c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 -- 172.21.15.50:0/3998519267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c109680 msgr2=0x7fd31c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c109680 0x7fd31c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 -- 172.21.15.50:0/3998519267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 msgr2=0x7fd31c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:48.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 -- 172.21.15.50:0/3998519267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c19b0a0 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319e00700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd319400700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 0x7fd31c1a0d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd30800e9c0 tx=0x7fd30800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800c9c0 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30800cb60 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c19b390 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.705+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308018680 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.706+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1a7660 con 0x7fd31c108d30 2024-05-10T21:19:48.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.707+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c0512c0 con 0x7fd31c108d30 2024-05-10T21:19:48.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.710+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd308010430 con 0x7fd31c108d30 2024-05-10T21:19:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.711+0000 7fd312a00700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 0x7fd3000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T21:19:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.711+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7894+0+0 (secure 0 0 0) 0x7fd3080e2510 con 0x7fd31c108d30 2024-05-10T21:19:48.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.712+0000 7fd319e00700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 0x7fd3000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T21:19:48.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.712+0000 7fd319e00700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 0x7fd3000b4e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd31c19bb30 tx=0x7fd30c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T21:19:48.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.714+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3080aa330 con 0x7fd31c108d30 2024-05-10T21:19:48.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:48.955+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd31c02e5f0 con 0x7fd31c108d30 2024-05-10T21:19:49.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.126+0000 7fd312a00700 1 -- 172.21.15.50:0/3998519267 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7fd3080b23b0 con 0x7fd31c108d30 2024-05-10T21:19:49.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.133+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 msgr2=0x7fd3000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:49.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.133+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 0x7fd3000b4e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd31c19bb30 tx=0x7fd30c009260 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.133+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 msgr2=0x7fd31c1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.133+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 0x7fd31c1a0d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd30800e9c0 tx=0x7fd30800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 shutdown_connections 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:6800/628586267,v1:172.21.15.50:6801/628586267] conn(0x7fd3000b2900 0x7fd3000b4e10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd31c104400 0x7fd31c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd31c109680 0x7fd31c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 --2- 172.21.15.50:0/3998519267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd31c108d30 0x7fd31c1a0d10 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd30800e9c0 tx=0x7fd30800ecd0 comp rx=0 tx=0).stop 2024-05-10T21:19:49.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.134+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 >> 172.21.15.50:0/3998519267 conn(0x7fd31c079d10 msgr2=0x7fd31c101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T21:19:49.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.136+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 shutdown_connections 2024-05-10T21:19:49.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-10T21:19:49.136+0000 7fd31b200700 1 -- 172.21.15.50:0/3998519267 wait complete. 2024-05-10T21:19:49.137 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-05-10T21:19:49.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:49 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:49.374 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:49 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:49.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:49 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T21:19:49.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:19:50.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:19:50.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:19:50.137 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T21:19:50.137 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T21:19:50.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 ceph-mon[29546]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:50.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 ceph-mon[29546]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T21:19:50.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 ceph-mon[29546]: pgmap v794: 1 pgs: 1 active+clean; 577 KiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:50.195 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T21:19:50.211 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T21:19:50.227 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T21:19:50.228 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T21:19:50.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a 2024-05-10T21:19:50.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:50 smithi092 ceph-mon[34415]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:50.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:50 smithi092 ceph-mon[34415]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T21:19:50.364 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:50 smithi092 ceph-mon[34415]: pgmap v794: 1 pgs: 1 active+clean; 577 KiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:50.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:50 smithi142 ceph-mon[34846]: from='client.? 172.21.15.50:0/3998519267' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T21:19:50.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:50 smithi142 ceph-mon[34846]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T21:19:50.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:50 smithi142 ceph-mon[34846]: pgmap v794: 1 pgs: 1 active+clean; 577 KiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T21:19:50.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 systemd[1]: Stopping Ceph mon.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29542]: 2024-05-10T21:19:50.884+0000 7f28aa800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:19:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a[29542]: 2024-05-10T21:19:50.884+0000 7f28aa800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T21:19:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:50 smithi050 podman[109240]: 2024-05-10 21:19:50.978321939 +0000 UTC m=+0.239272714 container died 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:19:51.649 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:51 smithi050 podman[109240]: 2024-05-10 21:19:51.450923126 +0000 UTC m=+0.711873894 container cleanup 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:19:51.649 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:51 smithi050 bash[109240]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a 2024-05-10T21:19:51.913 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 10 21:19:51 smithi050 podman[109254]: 2024-05-10 21:19:51.651821769 +0000 UTC m=+0.654811472 container remove 1ba2cba890c2de05772d97097ce383b18df768fa81d57fb2c02cd6cdf35ddd08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:19:52.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T21:19:52.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:52.321 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T21:19:52.321 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T21:19:52.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.b 2024-05-10T21:19:52.614 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:52 smithi092 systemd[1]: Stopping Ceph mon.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:52.879 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b[34411]: 2024-05-10T21:19:52.629+0000 7f2e402ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:19:52.879 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b[34411]: 2024-05-10T21:19:52.629+0000 7f2e402ad700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T21:19:53.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:52 smithi092 podman[69961]: 2024-05-10 21:19:52.885052098 +0000 UTC m=+0.367771486 container died a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:19:53.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 podman[69961]: 2024-05-10 21:19:53.30514968 +0000 UTC m=+0.787869066 container cleanup a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-10T21:19:53.534 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 bash[69961]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b 2024-05-10T21:19:53.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T21:19:53.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 podman[69975]: 2024-05-10 21:19:53.539817681 +0000 UTC m=+0.655053638 container remove a3aacc932add81ed493b3ba97bff73b089a427602417f93d8e5c9ca64f5d5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T21:19:53.785 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 systemd[1]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-10T21:19:53.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 systemd[1]: Stopped Ceph mon.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968. 2024-05-10T21:19:53.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 10 21:19:53 smithi092 systemd[1]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.b.service: Consumed 15.629s CPU time. 2024-05-10T21:19:53.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:53.816 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T21:19:53.816 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T21:19:53.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.c 2024-05-10T21:19:54.100 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:53 smithi142 systemd[1]: Stopping Ceph mon.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:54.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c[34841]: 2024-05-10T21:19:54.104+0000 7fb60cf12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:19:54.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c[34841]: 2024-05-10T21:19:54.104+0000 7fb60cf12700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T21:19:54.375 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:54 smithi142 podman[70741]: 2024-05-10 21:19:54.258123898 +0000 UTC m=+0.241201341 container died cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-10T21:19:54.834 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:54 smithi142 podman[70741]: 2024-05-10 21:19:54.554632092 +0000 UTC m=+0.537709535 container cleanup cdfeffead09cc4481619ea54c58c3f6d4062c3ad353dc8c0360e4081ca387151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:19:54.834 INFO:journalctl@ceph.mon.c.smithi142.stdout:May 10 21:19:54 smithi142 bash[70741]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mon-c 2024-05-10T21:19:54.971 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T21:19:55.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:55.013 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T21:19:55.013 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T21:19:55.013 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x 2024-05-10T21:19:55.413 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:55 smithi050 systemd[1]: Stopping Ceph mgr.x for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:55.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:19:55.562+0000 7fc3c2c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:19:55.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x[29777]: 2024-05-10T21:19:55.562+0000 7fc3c2c00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-10T21:19:55.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:55 smithi050 podman[109361]: 2024-05-10 21:19:55.668758844 +0000 UTC m=+0.257659535 container died 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:19:56.336 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:56 smithi050 podman[109361]: 2024-05-10 21:19:56.130655663 +0000 UTC m=+0.719556302 container cleanup 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-10T21:19:56.336 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:56 smithi050 bash[109361]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x 2024-05-10T21:19:56.663 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 10 21:19:56 smithi050 podman[109375]: 2024-05-10 21:19:56.340605833 +0000 UTC m=+0.655242777 container remove 806c29320aa8c45ba227b71c2560bde21bc461f241a03b91ae009d3a6a70e881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b710596db3f44de0a4b72eff459db857656fe05, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD) 2024-05-10T21:19:56.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T21:19:56.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:56.986 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T21:19:56.986 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T21:19:56.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.y 2024-05-10T21:19:57.342 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 21:19:57 smithi092 systemd[1]: Stopping Ceph mgr.y for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:57.614 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 21:19:57 smithi092 podman[70084]: 2024-05-10 21:19:57.347659145 +0000 UTC m=+0.161489948 container died 8eef36b1a8201205fb4ec92a8ed4926ee55c89c566fcf93862dee268cd1937e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:19:57.976 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 21:19:57 smithi092 podman[70084]: 2024-05-10 21:19:57.652632319 +0000 UTC m=+0.466463121 container cleanup 8eef36b1a8201205fb4ec92a8ed4926ee55c89c566fcf93862dee268cd1937e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:19:57.976 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 10 21:19:57 smithi092 bash[70084]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-y 2024-05-10T21:19:58.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T21:19:58.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:58.160 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T21:19:58.160 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T21:19:58.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.z 2024-05-10T21:19:58.466 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 21:19:58 smithi142 systemd[1]: Stopping Ceph mgr.z for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:19:58.735 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 21:19:58 smithi142 podman[70863]: 2024-05-10 21:19:58.47204101 +0000 UTC m=+0.110805862 container died f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-10T21:19:58.988 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 21:19:58 smithi142 podman[70863]: 2024-05-10 21:19:58.817867397 +0000 UTC m=+0.456632239 container cleanup f24f09d898e311c3945db95da5f71462057a059cdd52bad593a457f58c6a5fd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-10T21:19:58.988 INFO:journalctl@ceph.mgr.z.smithi142.stdout:May 10 21:19:58 smithi142 bash[70863]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mgr-z 2024-05-10T21:19:59.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T21:19:59.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:19:59.232 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T21:19:59.232 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T21:19:59.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.0 2024-05-10T21:19:59.663 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:19:59 smithi050 systemd[1]: Stopping Ceph osd.0 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:00.164 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:19:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0[42010]: 2024-05-10T21:19:59.782+0000 7fa479800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:00.164 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:19:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0[42010]: 2024-05-10T21:19:59.782+0000 7fa479800700 -1 osd.0 87 *** Got signal Terminated *** 2024-05-10T21:20:00.164 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:19:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0[42010]: 2024-05-10T21:19:59.782+0000 7fa479800700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:05.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:05 smithi050 podman[109480]: 2024-05-10 21:20:05.33864427 +0000 UTC m=+5.681954268 container died 12c163af1298ee659866ca3a32167bf9ab42da0da113833ac075d9db732c90cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:06.032 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:05 smithi050 podman[109480]: 2024-05-10 21:20:05.80185397 +0000 UTC m=+6.145163997 container cleanup 12c163af1298ee659866ca3a32167bf9ab42da0da113833ac075d9db732c90cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-10T21:20:06.032 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:05 smithi050 bash[109480]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0 2024-05-10T21:20:06.413 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:06 smithi050 podman[109502]: 2024-05-10 21:20:06.036440734 +0000 UTC m=+0.682347068 container remove 12c163af1298ee659866ca3a32167bf9ab42da0da113833ac075d9db732c90cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-10T21:20:07.163 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:06 smithi050 podman[109594]: 2024-05-10 21:20:06.800917159 +0000 UTC m=+0.040891330 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:07.163 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:06 smithi050 podman[109594]: 2024-05-10 21:20:06.903988116 +0000 UTC m=+0.143962284 container create 66d609da4761fc4cc7f4b125a92476aacc3c4cf249ac8bc5f88c448a210b1629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-10T21:20:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:07 smithi050 podman[109594]: 2024-05-10 21:20:07.182839532 +0000 UTC m=+0.422813734 container init 66d609da4761fc4cc7f4b125a92476aacc3c4cf249ac8bc5f88c448a210b1629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-10T21:20:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:07 smithi050 podman[109594]: 2024-05-10 21:20:07.195270599 +0000 UTC m=+0.435244801 container start 66d609da4761fc4cc7f4b125a92476aacc3c4cf249ac8bc5f88c448a210b1629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-10T21:20:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:07 smithi050 podman[109594]: 2024-05-10 21:20:07.331620413 +0000 UTC m=+0.571594679 container attach 66d609da4761fc4cc7f4b125a92476aacc3c4cf249ac8bc5f88c448a210b1629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-05-10T21:20:08.016 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 10 21:20:07 smithi050 podman[109594]: 2024-05-10 21:20:07.669545351 +0000 UTC m=+0.909519569 container died 66d609da4761fc4cc7f4b125a92476aacc3c4cf249ac8bc5f88c448a210b1629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:20:08.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T21:20:08.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:08.624 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T21:20:08.624 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T21:20:08.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.1 2024-05-10T21:20:08.927 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:08 smithi092 systemd[1]: Stopping Ceph osd.1 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:09.364 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1[39012]: 2024-05-10T21:20:08.931+0000 7fd2429d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:09.364 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1[39012]: 2024-05-10T21:20:08.931+0000 7fd2429d5700 -1 osd.1 87 *** Got signal Terminated *** 2024-05-10T21:20:09.364 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1[39012]: 2024-05-10T21:20:08.931+0000 7fd2429d5700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:14.526 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:14 smithi092 podman[70225]: 2024-05-10 21:20:14.271595011 +0000 UTC m=+5.445708541 container died 6d4e9ac5f5b9b738fc29876e7d12b7bac5a1c54a70135e1b85de8699895428f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True) 2024-05-10T21:20:14.864 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:14 smithi092 podman[70225]: 2024-05-10 21:20:14.631419238 +0000 UTC m=+5.805532767 container cleanup 6d4e9ac5f5b9b738fc29876e7d12b7bac5a1c54a70135e1b85de8699895428f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:14.864 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:14 smithi092 bash[70225]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1 2024-05-10T21:20:15.251 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:14 smithi092 podman[70242]: 2024-05-10 21:20:14.933166412 +0000 UTC m=+0.654310215 container remove 6d4e9ac5f5b9b738fc29876e7d12b7bac5a1c54a70135e1b85de8699895428f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-10T21:20:15.580 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.159749063 +0000 UTC m=+0.018185544 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:15.580 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.267234144 +0000 UTC m=+0.125670632 container create 39aae11a5e948a2087da4ffda47071c0dc5a10f1441ae6e13f2fec7c0710c359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-10T21:20:15.580 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.502907398 +0000 UTC m=+0.361343877 container init 39aae11a5e948a2087da4ffda47071c0dc5a10f1441ae6e13f2fec7c0710c359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True) 2024-05-10T21:20:15.581 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.506049983 +0000 UTC m=+0.364486472 container start 39aae11a5e948a2087da4ffda47071c0dc5a10f1441ae6e13f2fec7c0710c359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , ceph=True) 2024-05-10T21:20:15.581 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.586324803 +0000 UTC m=+0.444761294 container attach 39aae11a5e948a2087da4ffda47071c0dc5a10f1441ae6e13f2fec7c0710c359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:20:15.864 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 10 21:20:15 smithi092 podman[70334]: 2024-05-10 21:20:15.804750317 +0000 UTC m=+0.663186794 container died 39aae11a5e948a2087da4ffda47071c0dc5a10f1441ae6e13f2fec7c0710c359 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:16.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T21:20:16.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:16.469 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T21:20:16.469 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T21:20:16.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.2 2024-05-10T21:20:16.790 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:16 smithi142 systemd[1]: Stopping Ceph osd.2 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:17.125 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2[39998]: 2024-05-10T21:20:16.795+0000 7f98eaf7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:17.125 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2[39998]: 2024-05-10T21:20:16.795+0000 7f98eaf7e700 -1 osd.2 87 *** Got signal Terminated *** 2024-05-10T21:20:17.125 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2[39998]: 2024-05-10T21:20:16.795+0000 7f98eaf7e700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:22.625 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:22 smithi142 podman[70988]: 2024-05-10 21:20:22.348772831 +0000 UTC m=+5.673567410 container died abe5f581c3dc055b5cc747484c4b1d166d62b53fb4daf523c30976140755fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:23.124 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:22 smithi142 podman[70988]: 2024-05-10 21:20:22.671040565 +0000 UTC m=+5.995835139 container cleanup abe5f581c3dc055b5cc747484c4b1d166d62b53fb4daf523c30976140755fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-10T21:20:23.125 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:22 smithi142 bash[70988]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2 2024-05-10T21:20:23.125 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:22 smithi142 podman[71017]: 2024-05-10 21:20:22.888973805 +0000 UTC m=+0.542855586 container remove abe5f581c3dc055b5cc747484c4b1d166d62b53fb4daf523c30976140755fc9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T21:20:23.620 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.197941026 +0000 UTC m=+0.017398994 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:23.621 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.31464839 +0000 UTC m=+0.134106343 container create 959d30de968b5e5beb5095cbf36d73a1bb9ae67bd661481bc4feb7701070f76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-10T21:20:23.621 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.533874491 +0000 UTC m=+0.353332443 container init 959d30de968b5e5beb5095cbf36d73a1bb9ae67bd661481bc4feb7701070f76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-10T21:20:23.621 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.536866613 +0000 UTC m=+0.356324565 container start 959d30de968b5e5beb5095cbf36d73a1bb9ae67bd661481bc4feb7701070f76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-10T21:20:23.621 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.626364771 +0000 UTC m=+0.445822734 container attach 959d30de968b5e5beb5095cbf36d73a1bb9ae67bd661481bc4feb7701070f76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:23.875 INFO:journalctl@ceph.osd.2.smithi142.stdout:May 10 21:20:23 smithi142 podman[71109]: 2024-05-10 21:20:23.825550193 +0000 UTC m=+0.645008140 container died 959d30de968b5e5beb5095cbf36d73a1bb9ae67bd661481bc4feb7701070f76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:24.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T21:20:24.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:24.450 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T21:20:24.451 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T21:20:24.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.3 2024-05-10T21:20:24.914 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:24 smithi050 systemd[1]: Stopping Ceph osd.3 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:25.288 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:25.044+0000 7f305d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:25.288 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:25.044+0000 7f305d200700 -1 osd.3 87 *** Got signal Terminated *** 2024-05-10T21:20:25.289 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:25.044+0000 7f305d200700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:25.913 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:25 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:25.871+0000 7f3058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:19:59.758644+0000 front 2024-05-10T21:19:59.758818+0000 (oldest deadline 2024-05-10T21:20:25.658971+0000) 2024-05-10T21:20:26.581 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:26 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:26.271+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:27.163 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:26 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:26.905+0000 7f3058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:19:59.758644+0000 front 2024-05-10T21:19:59.758818+0000 (oldest deadline 2024-05-10T21:20:25.658971+0000) 2024-05-10T21:20:27.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:27.297+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:27.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:27.074+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:27.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:27 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:27.316+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:27.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:27 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:27.247+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:27.979 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:27 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:27.912+0000 7f3058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:19:59.758644+0000 front 2024-05-10T21:19:59.758818+0000 (oldest deadline 2024-05-10T21:20:25.658971+0000) 2024-05-10T21:20:28.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:28.070+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:28.340+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:28.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:28 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:28.273+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:28.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:28 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:28.284+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:29.163 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:28 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:28.909+0000 7f3058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:19:59.758644+0000 front 2024-05-10T21:19:59.758818+0000 (oldest deadline 2024-05-10T21:20:25.658971+0000) 2024-05-10T21:20:29.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:29.047+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:29.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:29.358+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:29.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:29 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:29.288+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:29.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:29 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:29.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:29.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:29.380+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:30.163 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:29 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3[48908]: 2024-05-10T21:20:29.891+0000 7f3058a00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:19:59.758644+0000 front 2024-05-10T21:19:59.758818+0000 (oldest deadline 2024-05-10T21:20:25.658971+0000) 2024-05-10T21:20:30.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:30.003+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:30.590 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:30 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:30.320+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:30.591 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:30 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:30.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:30.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:30 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:30.356+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:30.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:30.420+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:30.905 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:30 smithi050 podman[109699]: 2024-05-10 21:20:30.595746929 +0000 UTC m=+5.671309358 container died 30a31a843e5eed46aa1f55d12321cf7cccacd6c2b22a495854c9691356a3790e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-10T21:20:31.163 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:31 smithi050 podman[109699]: 2024-05-10 21:20:31.121456595 +0000 UTC m=+6.197019055 container cleanup 30a31a843e5eed46aa1f55d12321cf7cccacd6c2b22a495854c9691356a3790e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T21:20:31.164 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:31 smithi050 bash[109699]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3 2024-05-10T21:20:31.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:31.016+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:31.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:31 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:31.315+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:31.663 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:31 smithi050 podman[109730]: 2024-05-10 21:20:31.397958047 +0000 UTC m=+0.782239767 container remove 30a31a843e5eed46aa1f55d12321cf7cccacd6c2b22a495854c9691356a3790e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:31.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:31 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:31.282+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:31.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:31 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:31.237+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:31.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:31.407+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:32.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:32.064+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:32.568 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:32 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:32.240+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:32.568 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:32 smithi050 podman[109821]: 2024-05-10 21:20:32.245664224 +0000 UTC m=+0.038832287 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:32.568 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:32 smithi050 podman[109821]: 2024-05-10 21:20:32.382809902 +0000 UTC m=+0.175977991 container create 446f406e5c84d4759e21d89525c4b63dd04c600e9aee43c3ae2ef1b91d63d831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:20:32.569 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:32 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:32.199+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:32.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:32 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:32.284+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:32.830 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:32 smithi050 podman[109821]: 2024-05-10 21:20:32.711289222 +0000 UTC m=+0.504457334 container init 446f406e5c84d4759e21d89525c4b63dd04c600e9aee43c3ae2ef1b91d63d831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-10T21:20:32.830 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:32 smithi050 podman[109821]: 2024-05-10 21:20:32.724126632 +0000 UTC m=+0.517294721 container start 446f406e5c84d4759e21d89525c4b63dd04c600e9aee43c3ae2ef1b91d63d831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-10T21:20:32.830 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:32 smithi050 podman[109821]: 2024-05-10 21:20:32.835220221 +0000 UTC m=+0.628388366 container attach 446f406e5c84d4759e21d89525c4b63dd04c600e9aee43c3ae2ef1b91d63d831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:20:32.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:32.410+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:33.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:33.091+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:33.596 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:33.270+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:33.597 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 10 21:20:33 smithi050 podman[109821]: 2024-05-10 21:20:33.200943252 +0000 UTC m=+0.994111334 container died 446f406e5c84d4759e21d89525c4b63dd04c600e9aee43c3ae2ef1b91d63d831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True) 2024-05-10T21:20:33.597 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:33 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:33.221+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:33.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:33 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:33.268+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:33.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:33.411+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:33.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T21:20:34.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:34.052 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T21:20:34.052 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T21:20:34.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.4 2024-05-10T21:20:34.353 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:34 smithi092 systemd[1]: Stopping Ceph osd.4 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:34.353 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:34 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:34.229+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:34.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:34.077+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:34.613 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:34 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:34.358+0000 7f22e478d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:34.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:34 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:34.358+0000 7f22e478d700 -1 osd.4 87 *** Got signal Terminated *** 2024-05-10T21:20:34.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:34 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:34.358+0000 7f22e478d700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:34.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:34.240+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:34.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:34 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:34.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:34.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:34.443+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:35.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:35.112+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:35.613 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:35 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:35.388+0000 7f22dcd3b700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:08.061077+0000 front 2024-05-10T21:20:08.061120+0000 (oldest deadline 2024-05-10T21:20:35.061678+0000) 2024-05-10T21:20:35.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:35 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:35.186+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:35.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:35.233+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:35.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:35.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:35.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:35 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:35.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:35.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:35.420+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:35.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:35.420+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:36.374 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:36.128+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:36.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:36.272+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:36.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:36.217+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:36.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:36 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:36.217+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:36.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:36.220+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:36.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:36 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:36.359+0000 7f22dcd3b700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:08.061077+0000 front 2024-05-10T21:20:08.061120+0000 (oldest deadline 2024-05-10T21:20:35.061678+0000) 2024-05-10T21:20:36.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:36.425+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:36.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:36.425+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:37.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:37.044+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:37.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:37.147+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:37.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:37.147+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:37.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:37 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:37.379+0000 7f22dcd3b700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:08.061077+0000 front 2024-05-10T21:20:08.061120+0000 (oldest deadline 2024-05-10T21:20:35.061678+0000) 2024-05-10T21:20:37.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:37 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:37.231+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:37.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:37 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:37.231+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:37.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:37.231+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:37.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:37.179+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:37.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:37 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:37.179+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:37.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:37.428+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:37.877 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:37 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:37.428+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:38.119 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:37 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:37.854+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:38.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:38 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:38.034+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:38.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:38 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:38.174+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:38.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:38 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:38.174+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:38.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:38 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:38.194+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:38.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:38 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:38.159+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:38.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:38 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:38.159+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:38.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:38 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4[45690]: 2024-05-10T21:20:38.415+0000 7f22dcd3b700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:08.061077+0000 front 2024-05-10T21:20:08.061120+0000 (oldest deadline 2024-05-10T21:20:35.061678+0000) 2024-05-10T21:20:38.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:38 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:38.200+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:38.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:38 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:38.200+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:38.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:38 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:38.409+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:38.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:38 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:38.409+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:39.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:38 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:38.816+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:39.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:39.005+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:39.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:39.172+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:39.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:39.172+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:39.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:39 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:39.149+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:39.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:39 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:39.179+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:39.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:39 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:39.179+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:39.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:39 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:39.247+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:39.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:39 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:39.247+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:39.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:39.443+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:39.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:39.443+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:40.003 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:39 smithi092 podman[70457]: 2024-05-10 21:20:39.719951607 +0000 UTC m=+5.471764521 container died 014aba315de478d2e45857a4b9f01efca35f2267d1e647ddd04931caf26c9d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:40.004 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:39 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:39.812+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:40.328 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70457]: 2024-05-10 21:20:40.108402597 +0000 UTC m=+5.860215516 container cleanup 014aba315de478d2e45857a4b9f01efca35f2267d1e647ddd04931caf26c9d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:40.329 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 bash[70457]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4 2024-05-10T21:20:40.329 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:40 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:40.218+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:40.329 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:40 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:40.218+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:40.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:39 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:39.973+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:40.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:40 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:40.173+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:40.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:40 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:40.173+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:40.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:40.189+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:40.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:40.189+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:40.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:40.151+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:40.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:40.151+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:40.614 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70474]: 2024-05-10 21:20:40.334808981 +0000 UTC m=+0.615286429 container remove 014aba315de478d2e45857a4b9f01efca35f2267d1e647ddd04931caf26c9d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-10T21:20:40.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:40 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:40.474+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:40.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:40 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:40.474+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:40.965 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:40 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:40.859+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:40.965 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70567]: 2024-05-10 21:20:40.548968953 +0000 UTC m=+0.017932293 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:40.966 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70567]: 2024-05-10 21:20:40.660706343 +0000 UTC m=+0.129669655 container create c6adc6db4d0885927cf5e26e96010704ad377a5f17d226f3effa5c0c01c47529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-10T21:20:40.966 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70567]: 2024-05-10 21:20:40.888843307 +0000 UTC m=+0.357806629 container init c6adc6db4d0885927cf5e26e96010704ad377a5f17d226f3effa5c0c01c47529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:20:40.966 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70567]: 2024-05-10 21:20:40.892211775 +0000 UTC m=+0.361175088 container start c6adc6db4d0885927cf5e26e96010704ad377a5f17d226f3effa5c0c01c47529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:40.966 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:40 smithi092 podman[70567]: 2024-05-10 21:20:40.971364551 +0000 UTC m=+0.440327903 container attach c6adc6db4d0885927cf5e26e96010704ad377a5f17d226f3effa5c0c01c47529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:20:41.236 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 10 21:20:41 smithi092 podman[70567]: 2024-05-10 21:20:41.186699225 +0000 UTC m=+0.655662555 container died c6adc6db4d0885927cf5e26e96010704ad377a5f17d226f3effa5c0c01c47529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:20:41.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:40 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:40.952+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:41.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:41 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:41.162+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:41.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:41 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:41.162+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:41.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:41 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:41.128+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:41.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:41 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:41.128+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:41.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:41 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:41.234+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:41.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:41 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:41.234+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:41.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:41 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:41.240+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:41.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:41 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:41.240+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:41.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T21:20:41.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:41.812 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T21:20:41.813 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T21:20:41.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.5 2024-05-10T21:20:41.838 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:41 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:41.510+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:41.838 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:41 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:41.510+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:42.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:41 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:41.812+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:42.119 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:41 smithi142 systemd[1]: Stopping Ceph osd.5 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:42.119 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:41 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:41.908+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:42.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:42.185+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:42.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:42.185+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:42.375 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:42.123+0000 7f27af00d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:42.375 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:42.123+0000 7f27af00d700 -1 osd.5 87 *** Got signal Terminated *** 2024-05-10T21:20:42.375 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:42.123+0000 7f27af00d700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:42.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:42 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:42.216+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:42.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:42 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:42.216+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:42.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:42 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:42.151+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:42.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:42 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:42.151+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:42.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:42 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:42.215+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:42.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:42 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:42.215+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:42.874 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:42.499+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:42.875 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:42.499+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:43.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:42 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:42.845+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:43.225 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:42 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:42.917+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:43.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:43 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:43.192+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:43.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:43 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:43.192+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:43.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:43 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:43.140+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:43.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:43 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:43.140+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:43.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:43 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:43.140+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:43.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:43 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:43.187+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:43.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:43 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:43.187+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:43.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:43 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:43.187+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:43.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:43.229+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:43.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:43.229+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:43.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:43.459+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:43.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:43.459+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:44.163 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:43 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:43.889+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:44.163 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:43 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:43.889+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:44.233 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:43 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:43.914+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:44.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:44.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:44.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:44.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:44.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:44.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:44.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:44.238+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:44.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:44.238+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:44.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:44.238+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:44.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:44.444+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:44.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:44.444+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:44.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:44 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:44.229+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:44.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:44 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:44.229+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:44.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:44 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:44.183+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:44.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:44 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:44.183+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:44.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:44 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:44.183+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:45.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:44.848+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:45.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:44 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:44.848+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:45.236 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:44 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:44.935+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:45.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:45 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:45.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:45.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:45 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:45.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:45.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:45 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:45.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:45.624 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:45.405+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:45.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:45.405+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:45.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:45.405+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:18.940486+0000 front 2024-05-10T21:20:18.940527+0000 (oldest deadline 2024-05-10T21:20:44.840629+0000) 2024-05-10T21:20:45.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:45.241+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:45.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:45.241+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:45.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:45.241+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:45.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:45.181+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:45.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:45.181+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:45.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:45.215+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:45.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:45.215+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:45.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:45 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:45.215+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:46.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:45 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:45.861+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:46.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:45 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:45.861+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:46.244 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:45 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:45.933+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:46.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:46.139+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:46.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:46.139+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:46.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:46.139+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:46.393+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.837080+0000 front 2024-05-10T21:20:02.837141+0000 (oldest deadline 2024-05-10T21:20:28.736815+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:46.393+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.639111+0000 front 2024-05-10T21:20:12.639201+0000 (oldest deadline 2024-05-10T21:20:34.939091+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5[46939]: 2024-05-10T21:20:46.393+0000 7f27ab5c3700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:18.940486+0000 front 2024-05-10T21:20:18.940527+0000 (oldest deadline 2024-05-10T21:20:44.840629+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:46.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:46.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:46.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:46.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:46.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:46.198+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:46.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:46.198+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:46.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:46.250+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:46.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:46.250+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:46.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:46.250+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:47.156 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:46.894+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:47.156 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:46 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:46.894+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:47.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:46 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:46.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:47.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:47 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:47.200+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:47.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:47 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:47.200+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:47.375 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:47 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:47.200+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:47.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:47 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:47.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:47.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:47 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:47.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:47.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:47 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:47.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:47.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:47.274+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:47.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:47.274+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:47.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:47.274+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:47.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:47.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:47.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:47.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:47.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:47 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:47.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:47.787 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:47 smithi142 podman[71218]: 2024-05-10 21:20:47.514307771 +0000 UTC m=+5.501274147 container died e24b7a4ff3f3cd72a3ef4aa8ad234073f46bf0575705055a541014001051ed68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:20:48.113 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:47 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:47.850+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:48.114 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:47 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:47.850+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:48.124 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:47 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:47.936+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:48.125 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:47 smithi142 podman[71218]: 2024-05-10 21:20:47.892828383 +0000 UTC m=+5.879794768 container cleanup e24b7a4ff3f3cd72a3ef4aa8ad234073f46bf0575705055a541014001051ed68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-10T21:20:48.125 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:47 smithi142 bash[71218]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5 2024-05-10T21:20:48.429 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71235]: 2024-05-10 21:20:48.143897442 +0000 UTC m=+0.624167794 container remove e24b7a4ff3f3cd72a3ef4aa8ad234073f46bf0575705055a541014001051ed68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-10T21:20:48.429 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:48 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:48.214+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:48.429 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:48 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:48.214+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:48.429 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:48 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:48.214+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:48.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:48 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:48.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:48.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:48 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:48.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:48.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:48 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:48.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:48.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:48.279+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:48.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:48.279+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:48.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:48.279+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:48.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:48.281+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:48.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:48.281+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:48.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:48 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:48.281+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:48.750 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.337986821 +0000 UTC m=+0.017898612 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:48.751 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.478185157 +0000 UTC m=+0.158113943 container create 41d14d7a3a91c0ba5602e2d729f23ac28719428ed4c3aa049bf206a5d2e172c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T21:20:48.751 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.681539616 +0000 UTC m=+0.361451388 container init 41d14d7a3a91c0ba5602e2d729f23ac28719428ed4c3aa049bf206a5d2e172c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:20:48.751 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.684624208 +0000 UTC m=+0.364535981 container start 41d14d7a3a91c0ba5602e2d729f23ac28719428ed4c3aa049bf206a5d2e172c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-10T21:20:48.751 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.756398693 +0000 UTC m=+0.436310489 container attach 41d14d7a3a91c0ba5602e2d729f23ac28719428ed4c3aa049bf206a5d2e172c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD) 2024-05-10T21:20:49.125 INFO:journalctl@ceph.osd.5.smithi142.stdout:May 10 21:20:48 smithi142 podman[71325]: 2024-05-10 21:20:48.983759302 +0000 UTC m=+0.663671067 container died 41d14d7a3a91c0ba5602e2d729f23ac28719428ed4c3aa049bf206a5d2e172c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-10T21:20:49.125 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:48 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:48.971+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:49.125 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:48 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:48.971+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:49.135 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:48 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:48.872+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:49.135 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:48 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:48.872+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:49.537 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:49 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:49.253+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:49.537 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:49 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:49.254+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:49.537 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:49 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:49.254+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:49.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T21:20:49.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:49 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:49.140+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:49.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:49 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:49.140+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:49.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:49 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:49.140+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:49.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:49.643 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T21:20:49.643 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T21:20:49.644 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.6 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:49.291+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:49.291+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:49.291+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:49.260+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:49.260+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:49.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:49 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:49.260+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:50.163 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:49 smithi050 systemd[1]: Stopping Ceph osd.6 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:50.254 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:49 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:49.970+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:50.254 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:49 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:49.970+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:50.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:49 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:49.875+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:50.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:49 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:49.875+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:50.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:50.116+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:50.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:50.116+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:50.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:50.116+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:50.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:50 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:50.259+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:50.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:50 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:50.259+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:50.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:50 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:50.259+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:50.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:50.268+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:50.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:50.268+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:50.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:50.268+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.232+0000 7f829ee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.232+0000 7f829ee00700 -1 osd.6 87 *** Got signal Terminated *** 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.232+0000 7f829ee00700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.233+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.233+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:50.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:50 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:50.233+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:51.286 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:50 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:50.944+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:51.286 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:50 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:50.944+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:51.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:50.892+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:51.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:50 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:50.892+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:51.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:51 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:51.102+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:51.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:51 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:51.102+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:51.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:51 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:51.102+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:51.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:51 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:51.291+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:51.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:51 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:51.291+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:51.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:51 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:51.291+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:51.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:51.213+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:51.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:51.213+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:51.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:51.213+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:51.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:51.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:51.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:51.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:51.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:51.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:51.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:51 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:51.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:52.336 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:51 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:51.940+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:52.336 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:51 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:51.940+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:52.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:51 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:51.933+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:52.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:51 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:51.933+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:52.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:52.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:52.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:52.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:52.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:52.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:52.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:52.341+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:52.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:52.341+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:52.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:52.341+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:52.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:52.226+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:52.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:52.226+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:52.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:52.226+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:52.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:52.226+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.761499+0000 front 2024-05-10T21:20:29.761564+0000 (oldest deadline 2024-05-10T21:20:51.461570+0000) 2024-05-10T21:20:52.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:52.205+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:52.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:52.205+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:52.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:52.205+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:52.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:52.205+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:52.950+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:52.950+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:52 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:52.950+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:53.135+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:53.135+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:53.135+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:53.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:53.135+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:53.367 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:52.912+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:53.367 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:52.912+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:53.367 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:52 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:52.912+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:53.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:53.372+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:53.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:53.372+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:53.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:53.372+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:53.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:53.269+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:53.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:53.269+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:53.663 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:53.269+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:53.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:53.269+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.761499+0000 front 2024-05-10T21:20:29.761564+0000 (oldest deadline 2024-05-10T21:20:51.461570+0000) 2024-05-10T21:20:53.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:53.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:53.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:53.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:53.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:53.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:53.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:53 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:53.233+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:53.947+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:53.947+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:53 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:53.947+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:54.184+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:54.184+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:54.184+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:54.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:54.184+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:54.372 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:53.885+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:54.372 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:53.885+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:54.372 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:53 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:53.885+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:54.565 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:54.228+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:54.565 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:54.228+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:54.565 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:54.228+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:54.565 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:54.228+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:54.566 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:54.257+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.550506+0000 front 2024-05-10T21:20:02.551194+0000 (oldest deadline 2024-05-10T21:20:26.550760+0000) 2024-05-10T21:20:54.566 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:54.257+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:11.753706+0000 front 2024-05-10T21:20:11.753787+0000 (oldest deadline 2024-05-10T21:20:39.355218+0000) 2024-05-10T21:20:54.566 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:54.257+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.356734+0000 front 2024-05-10T21:20:19.356567+0000 (oldest deadline 2024-05-10T21:20:42.758033+0000) 2024-05-10T21:20:54.566 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:54 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6[55436]: 2024-05-10T21:20:54.257+0000 7f829a600700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.761499+0000 front 2024-05-10T21:20:29.761564+0000 (oldest deadline 2024-05-10T21:20:51.461570+0000) 2024-05-10T21:20:54.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:54.377+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:54.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:54.377+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:54.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:54.377+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:55.353 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:54.934+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:55.353 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:54.934+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:55.354 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:54 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:54.934+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:55.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:54.961+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:55.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:54.961+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:55.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:54 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:54.961+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:55.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:55.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:55.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:55.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:55.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:55.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:55.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:55.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:55.597 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:55.253+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:55.597 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:55.253+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:55.597 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:55.253+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:55.597 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:55.253+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:55.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:55.358+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:55.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:55.358+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:55.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:55.358+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:55.913 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:55 smithi050 podman[109926]: 2024-05-10 21:20:55.817181209 +0000 UTC m=+5.728445228 container died 46152bead5fffaa4d12f8b4cbc88fbae67c79d929dbe3719c5c4b948a26f9a6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:20:56.232 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:56 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:56.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:56.232 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:56 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:56.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:56.232 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:56 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:56.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:56.232 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:56 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:56.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:55.959+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:55.959+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:55 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:55.959+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:56 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:56.150+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:56 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:56.150+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:56 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:56.150+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:56.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:56 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:56.150+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:56.365 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:55.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:56.365 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:55.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:56.365 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:55 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:55.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:56.512 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:56 smithi050 podman[109926]: 2024-05-10 21:20:56.241689666 +0000 UTC m=+6.152953705 container cleanup 46152bead5fffaa4d12f8b4cbc88fbae67c79d929dbe3719c5c4b948a26f9a6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-10T21:20:56.513 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:56 smithi050 bash[109926]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6 2024-05-10T21:20:56.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:56.370+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:56.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:56.370+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:56.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:56.370+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:56.913 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:56 smithi050 podman[109945]: 2024-05-10 21:20:56.517544013 +0000 UTC m=+0.687779992 container remove 46152bead5fffaa4d12f8b4cbc88fbae67c79d929dbe3719c5c4b948a26f9a6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:57.001+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:57.001+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:57.001+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:57.172+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:57.172+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:57.172+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:57.265 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:57 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:57.172+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:57.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:56.996+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:57.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:56.996+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:57.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:56 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:56.996+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:57.510 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:57.258+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:57.510 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:57.258+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:57.510 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:57.258+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:57.510 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:57 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:57.258+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:57.511 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:57 smithi050 podman[110040]: 2024-05-10 21:20:57.212122485 +0000 UTC m=+0.033583866 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:20:57.511 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:57 smithi050 podman[110040]: 2024-05-10 21:20:57.352164016 +0000 UTC m=+0.173625450 container create a1ad36414410e90c98eb19a66a306bde07a508790848ca2cf9f77f17d7d55573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-10T21:20:57.874 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:57 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:57.380+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:57.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:57 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:57.380+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:57.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:57 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:57.380+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:57.914 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:57 smithi050 podman[110040]: 2024-05-10 21:20:57.630911389 +0000 UTC m=+0.452372836 container init a1ad36414410e90c98eb19a66a306bde07a508790848ca2cf9f77f17d7d55573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-10T21:20:57.914 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:57 smithi050 podman[110040]: 2024-05-10 21:20:57.642586494 +0000 UTC m=+0.464047907 container start a1ad36414410e90c98eb19a66a306bde07a508790848ca2cf9f77f17d7d55573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5) 2024-05-10T21:20:57.914 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:57 smithi050 podman[110040]: 2024-05-10 21:20:57.729677318 +0000 UTC m=+0.551138718 container attach a1ad36414410e90c98eb19a66a306bde07a508790848ca2cf9f77f17d7d55573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507) 2024-05-10T21:20:58.347 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:58.034+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:58.347 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:58.034+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:58.347 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:58.034+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:58.048+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:58.048+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:58.048+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:58.175+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:58.175+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:58.175+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:58.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:58.175+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:58.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 10 21:20:58 smithi050 podman[110040]: 2024-05-10 21:20:58.117047863 +0000 UTC m=+0.938509297 container died a1ad36414410e90c98eb19a66a306bde07a508790848ca2cf9f77f17d7d55573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True) 2024-05-10T21:20:58.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:58.292+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:58.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:58.293+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:58.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:58.293+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:58.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:58 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:58.293+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:20:58.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:58.352+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:58.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:58.352+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:58.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:58 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:58.352+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:58.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T21:20:59.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:20:59.002 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T21:20:59.002 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T21:20:59.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.7 2024-05-10T21:20:59.287 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.040+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:20:59.287 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.040+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:20:59.287 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.040+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:20:59.287 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 systemd[1]: Stopping Ceph osd.7 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:20:59.287 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:59.125+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:20:59.288 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:59.125+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:20:59.288 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:59.125+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:20:59.288 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:20:59.125+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:20:59.359 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:59.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:20:59.359 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:59.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:20:59.359 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:20:59.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:20:59.613 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.292+0000 7fc783ece700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:20:59.614 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.292+0000 7fc783ece700 -1 osd.7 87 *** Got signal Terminated *** 2024-05-10T21:20:59.614 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:20:59 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:20:59.292+0000 7fc783ece700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:20:59.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:59.361+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:20:59.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:59.361+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:20:59.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:20:59 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:20:59.361+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:20:59.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:59.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:20:59.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:59.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:20:59.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:59.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:20:59.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:20:59 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:20:59.257+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:00.362 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:00.021+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:00.362 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:00.021+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:00.362 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:00.021+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:00.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:00.021+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:21:00.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:00.021+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:21:00.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:00.021+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:21:00.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:00.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:00.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:00.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:00.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:00.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:00.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:00 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:00.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:00.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:00.367+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:00.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:00.367+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:00.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:00 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:00.367+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:00.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:00.300+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:00.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:00.300+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:00.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:00.300+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:00.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:00 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:00.300+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:01.064+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:01.064+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:01.064+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:01.177+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:01.177+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:01.177+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:01.177+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:01.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:01.177+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:01.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:01.014+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:01.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:01.014+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:01.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:01.014+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:01.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:01.308+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:01.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:01.308+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:01.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:01.308+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:01.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:01.308+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:01.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:01 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:01.308+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:01.874 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:01.395+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:01.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:01.395+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:01.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:01 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:01.395+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:02.038+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:02.038+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:02.038+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:02.188+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:02.188+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:02.188+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:02.188+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:02.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:02 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:02.188+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:02.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:02.026+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:02.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:02.026+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:02.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:02.026+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:02.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:02.309+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:02.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:02.309+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:02.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:02.309+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:02.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:02.309+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:02.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:02 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:02.309+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:02.874 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:02.425+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:02.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:02.425+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:02.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:02 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:02.425+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:03.059+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:03.059+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:03.059+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:03.204+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:03.204+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:03.204+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:03.204+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:03.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:03 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:03.204+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:03.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.008+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:03.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.008+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:03.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.008+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:03.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.008+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:03.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:03.274+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:03.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:03.274+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:03.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:03.274+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:03.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:03.274+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:03.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:03 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:03.274+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:03.874 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:03.431+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:03.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:03.431+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:03.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:03.431+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:03.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:03.431+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:04.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:04.094+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.017328+0000 front 2024-05-10T21:20:13.017276+0000 (oldest deadline 2024-05-10T21:20:37.717474+0000) 2024-05-10T21:21:04.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:04.094+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:19.418736+0000 front 2024-05-10T21:20:19.418562+0000 (oldest deadline 2024-05-10T21:20:42.918531+0000) 2024-05-10T21:21:04.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:04.094+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:29.319995+0000 front 2024-05-10T21:20:29.320137+0000 (oldest deadline 2024-05-10T21:20:52.720256+0000) 2024-05-10T21:21:04.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7[52403]: 2024-05-10T21:21:04.094+0000 7fc77c47c700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.421209+0000 front 2024-05-10T21:20:37.421247+0000 (oldest deadline 2024-05-10T21:21:03.321382+0000) 2024-05-10T21:21:04.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:04.199+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:04.365 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:04.199+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:04.365 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:04.199+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:04.365 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:04.199+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:04.365 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:04 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:04.199+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:04.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.986+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:04.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.986+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:04.374 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.986+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:04.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:03 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:03.986+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:04.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:04.231+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:04.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:04.231+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:04.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:04.231+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:04.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:04.231+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:04.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:04 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:04.231+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:04.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:04.402+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:04.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:04.402+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:04.876 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:04.402+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:04.876 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:04 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:04.402+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:04.946 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:04 smithi092 podman[70683]: 2024-05-10 21:21:04.667477211 +0000 UTC m=+5.472897764 container died b565d95be35dcb7d9bafd999ec622b4bc7063e53f086113d9f79f0ba4086b4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-10T21:21:05.329 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70683]: 2024-05-10 21:21:05.050823218 +0000 UTC m=+5.856243767 container cleanup b565d95be35dcb7d9bafd999ec622b4bc7063e53f086113d9f79f0ba4086b4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-10T21:21:05.329 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 bash[70683]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7 2024-05-10T21:21:05.330 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:05.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:05.330 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:05.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:05.330 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:05.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:05.330 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:05.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:05.330 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:05.154+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:05.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.035+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:05.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.035+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:05.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.035+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:05.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.035+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:05.613 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70703]: 2024-05-10 21:21:05.335271156 +0000 UTC m=+0.658770392 container remove b565d95be35dcb7d9bafd999ec622b4bc7063e53f086113d9f79f0ba4086b4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-10T21:21:05.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:05 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:05.187+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:05.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:05 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:05.187+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:05.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:05 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:05.187+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:05.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:05 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:05.187+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:05.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:05 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:05.187+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:05.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:05.418+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:05.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:05.418+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:05.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:05.418+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:05.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:05.418+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:05.908 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70793]: 2024-05-10 21:21:05.56614509 +0000 UTC m=+0.017821637 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:21:05.908 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70793]: 2024-05-10 21:21:05.669872839 +0000 UTC m=+0.121549428 container create 345d288ed7f3ff0552f4fa50945511a6334cb2a710a5134e36ce5ab4ba7bc00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5) 2024-05-10T21:21:05.908 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70793]: 2024-05-10 21:21:05.838710692 +0000 UTC m=+0.290387238 container init 345d288ed7f3ff0552f4fa50945511a6334cb2a710a5134e36ce5ab4ba7bc00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-10T21:21:05.908 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70793]: 2024-05-10 21:21:05.841814358 +0000 UTC m=+0.293490907 container start 345d288ed7f3ff0552f4fa50945511a6334cb2a710a5134e36ce5ab4ba7bc00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:21:05.909 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:05 smithi092 podman[70793]: 2024-05-10 21:21:05.913876989 +0000 UTC m=+0.365553538 container attach 345d288ed7f3ff0552f4fa50945511a6334cb2a710a5134e36ce5ab4ba7bc00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T21:21:06.363 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 10 21:21:06 smithi092 podman[70793]: 2024-05-10 21:21:06.132875868 +0000 UTC m=+0.584552408 container died 345d288ed7f3ff0552f4fa50945511a6334cb2a710a5134e36ce5ab4ba7bc00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:21:06.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:06 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:06.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:06.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:06 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:06.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:06.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:06 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:06.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:06.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:06 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:06.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:06.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:06 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:06.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:06.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.992+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:06.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.992+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:06.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.992+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:06.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:05 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:05.992+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:06.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:06 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:06.185+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:06.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:06 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:06.185+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:06.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:06 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:06.185+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:06.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:06 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:06.185+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:06.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:06 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:06.186+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:06.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T21:21:06.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:06.773 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T21:21:06.773 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T21:21:06.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.8 2024-05-10T21:21:06.798 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:06.440+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:06.799 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:06.440+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:06.799 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:06.440+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:06.799 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:06.440+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:07.081 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:06 smithi142 systemd[1]: Stopping Ceph osd.8 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:06.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:06.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:06.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:06 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:06.959+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:07.086+0000 7f37138b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:07.086+0000 7f37138b2700 -1 osd.8 87 *** Got signal Terminated *** 2024-05-10T21:21:07.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:07.086+0000 7f37138b2700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:21:07.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:07 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:07.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:07.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:07 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:07.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:07.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:07 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:07.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:07.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:07 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:07.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:07.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:07 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:07.087+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:07.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:07 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:07.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:07.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:07 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:07.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:07.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:07 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:07.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:07.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:07 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:07.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:07.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:07 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:07.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:07.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:07.391+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:07.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:07.391+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:07.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:07.391+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:07.875 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:07 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:07.391+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:08.357 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:08.001+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:08.357 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:08.001+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:08.357 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:08.001+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:08.357 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:08.001+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:08.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:08.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:08.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:08.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:08.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:08.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:08 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:08.104+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:08.413 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:08.156+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:08.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:08.156+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:08.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:08.156+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:08.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:08.156+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:08.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:08 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:08.156+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:08.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:08.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:08.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:08.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:08.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:08.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:08.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:08.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:08.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:08 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:08.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:09.361 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:09.017+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:09.606 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:09.017+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:09.606 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:09.017+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:09.606 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:09.017+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:09.606 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:09.017+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.157615+0000 front 2024-05-10T21:20:46.157712+0000 (oldest deadline 2024-05-10T21:21:08.457742+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.152+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.152+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:09.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:09.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:09.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:09.366+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:09.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:09.366+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:09.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:09.366+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:09.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:09.366+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:09.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:09 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:09.366+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:09.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:09.182+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:09.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:09.182+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:09.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:09.182+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:09.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:09.182+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:09.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:09 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:09.182+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:10.339 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:10.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:10.339 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:10.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:10.339 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:10.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:10.339 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:10.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:10.339 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:10.018+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.157615+0000 front 2024-05-10T21:20:46.157712+0000 (oldest deadline 2024-05-10T21:21:08.457742+0000) 2024-05-10T21:21:10.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:10.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:10.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:10.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:10.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:10.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:10 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:10.131+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:10.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:10.344+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:10.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:10.344+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:10.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:10.344+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:10.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:10.344+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:10.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:10 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:10.344+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:10.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:10.178+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:10.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:10.178+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:10.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:10.178+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:10.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:10.178+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:10.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:10 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:10.178+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:11.309 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:11.050+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:11.309 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:11.050+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:11.309 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:11.050+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:11.309 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:11.050+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:11.310 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:11.050+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.157615+0000 front 2024-05-10T21:20:46.157712+0000 (oldest deadline 2024-05-10T21:21:08.457742+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:11.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:11 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:11.180+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:11.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:11.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:11.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:11.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:11.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:11.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:11.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:11.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:11.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:11 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:11.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:11.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:11 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:11.222+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:12.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:12.085+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.852392+0000 front 2024-05-10T21:20:12.852419+0000 (oldest deadline 2024-05-10T21:20:36.952696+0000) 2024-05-10T21:21:12.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:12.085+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:16.953011+0000 front 2024-05-10T21:20:16.953166+0000 (oldest deadline 2024-05-10T21:20:48.153875+0000) 2024-05-10T21:21:12.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:12.085+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:28.154668+0000 front 2024-05-10T21:20:28.154490+0000 (oldest deadline 2024-05-10T21:20:52.254501+0000) 2024-05-10T21:21:12.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:12.085+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.056263+0000 front 2024-05-10T21:20:38.056131+0000 (oldest deadline 2024-05-10T21:21:02.756184+0000) 2024-05-10T21:21:12.375 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8[53520]: 2024-05-10T21:21:12.085+0000 7f370be60700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.157615+0000 front 2024-05-10T21:20:46.157712+0000 (oldest deadline 2024-05-10T21:21:08.457742+0000) 2024-05-10T21:21:12.376 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:12.281+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:12.376 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:12.281+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:12.376 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:12.281+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:12.376 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:12.281+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:12.376 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:12.281+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:12.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:12.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:12.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:12.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:12.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:12.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:12 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:12.137+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:12.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:12.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:12.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:12.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:12.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:12.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:12 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:12.209+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:12.799 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 podman[71431]: 2024-05-10 21:21:12.531130977 +0000 UTC m=+5.551127983 container died c55d5ab051cf49a4d45fb7a453d77bf1559eb27e735220be8d22341fb675e522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-10T21:21:13.124 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 podman[71431]: 2024-05-10 21:21:12.894803498 +0000 UTC m=+5.914800497 container cleanup c55d5ab051cf49a4d45fb7a453d77bf1559eb27e735220be8d22341fb675e522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-10T21:21:13.124 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:12 smithi142 bash[71431]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:13.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:13 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:13.112+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:13.507 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71448]: 2024-05-10 21:21:13.18793471 +0000 UTC m=+0.652078046 container remove c55d5ab051cf49a4d45fb7a453d77bf1559eb27e735220be8d22341fb675e522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-10T21:21:13.508 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:13 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:13.257+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:13.508 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:13 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:13.257+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:13.508 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:13 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:13.257+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:13.508 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:13 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:13.257+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:13.508 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:13 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:13.257+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:13.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:13.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:13.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:13.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:13.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:13.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:13 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:13.254+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:13.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71537]: 2024-05-10 21:21:13.416745071 +0000 UTC m=+0.018057517 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:21:13.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71537]: 2024-05-10 21:21:13.597156731 +0000 UTC m=+0.198469166 container create 27ebbc3efc9bfd29573b11b8608c6e7a928811815f2cf56232a2bdb9d716684e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-05-10T21:21:13.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71537]: 2024-05-10 21:21:13.79921142 +0000 UTC m=+0.400523848 container init 27ebbc3efc9bfd29573b11b8608c6e7a928811815f2cf56232a2bdb9d716684e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:21:14.090 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71537]: 2024-05-10 21:21:13.802322968 +0000 UTC m=+0.403635407 container start 27ebbc3efc9bfd29573b11b8608c6e7a928811815f2cf56232a2bdb9d716684e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-10T21:21:14.090 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:13 smithi142 podman[71537]: 2024-05-10 21:21:13.907543086 +0000 UTC m=+0.508855528 container attach 27ebbc3efc9bfd29573b11b8608c6e7a928811815f2cf56232a2bdb9d716684e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-10T21:21:14.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:14.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:14.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:14.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:14.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:14.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:14 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:14.100+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.8.smithi142.stdout:May 10 21:21:14 smithi142 podman[71537]: 2024-05-10 21:21:14.096240098 +0000 UTC m=+0.697552535 container died 27ebbc3efc9bfd29573b11b8608c6e7a928811815f2cf56232a2bdb9d716684e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:14 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:14.246+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:14 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:14.246+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:14 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:14.246+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:14 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:14.246+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:14.368 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:14 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:14.246+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:14.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:14.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:14.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:14.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:14.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:14.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:14.245+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:14.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T21:21:14.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:14.767 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T21:21:14.767 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T21:21:14.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.9 2024-05-10T21:21:15.163 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:14 smithi050 systemd[1]: Stopping Ceph osd.9 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:15.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:15 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:15.084+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:15.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:15 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:15.261+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:15.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:15 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:15.261+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:15.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:15 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:15.261+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:15.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:15 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:15.261+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:15.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:15 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:15.261+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:15.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.248+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.436+0000 7faba8000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.436+0000 7faba8000700 -1 osd.9 87 *** Got signal Terminated *** 2024-05-10T21:21:15.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:15 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:15.436+0000 7faba8000700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:21:16.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:16.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:16.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:16.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:16.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:16.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:16 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:16.059+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:16.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:16.295+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:16.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:16.295+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:16.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:16.295+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:16.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:16.295+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:16.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:16 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:16.295+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:16.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:16.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:16.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:16.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:16.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:16 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:16.236+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:17.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:17.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:17.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:17.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:17.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:17.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:17 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:17.097+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:17.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:17.314+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:17.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:17 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:17.219+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:18.363 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:18.364 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:18 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:18.110+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:18.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:18 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:18.338+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:18.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:18.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:18.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:18.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:18.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:18.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:18 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:18.238+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:19.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:19.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:19 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:19.132+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:19.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:19 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:19.305+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:19.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:19.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:19.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:19.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:19.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:19.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:19 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:19.223+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:20.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:20 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:20.120+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:20.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.324+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:20.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.324+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:20.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.324+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:20.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.325+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:20.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.325+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:20.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:20 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:20.325+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:20.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:02.905220+0000 front 2024-05-10T21:20:02.906392+0000 (oldest deadline 2024-05-10T21:20:25.805258+0000) 2024-05-10T21:21:20.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:13.409421+0000 front 2024-05-10T21:20:13.409695+0000 (oldest deadline 2024-05-10T21:20:35.109621+0000) 2024-05-10T21:21:20.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.913416+0000 front 2024-05-10T21:20:20.913338+0000 (oldest deadline 2024-05-10T21:20:46.213551+0000) 2024-05-10T21:21:20.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.716268+0000 front 2024-05-10T21:20:26.716059+0000 (oldest deadline 2024-05-10T21:20:50.815760+0000) 2024-05-10T21:21:20.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:39.018948+0000 front 2024-05-10T21:20:39.018869+0000 (oldest deadline 2024-05-10T21:21:00.619962+0000) 2024-05-10T21:21:20.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9[62479]: 2024-05-10T21:21:20.174+0000 7faba3800700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.422613+0000 front 2024-05-10T21:20:46.422549+0000 (oldest deadline 2024-05-10T21:21:10.522741+0000) 2024-05-10T21:21:21.277 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:20 smithi050 podman[110145]: 2024-05-10 21:21:20.990509082 +0000 UTC m=+5.739658696 container died f8f98545bf4cf22a20915474eabad5f95d985b74a84ca3455a7ae0a65b1e53e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:21.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:21 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:21.166+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:21.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:21.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:21.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:21.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:21.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:21.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:21.337+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:21.650 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:21 smithi050 podman[110145]: 2024-05-10 21:21:21.469053151 +0000 UTC m=+6.218202799 container cleanup f8f98545bf4cf22a20915474eabad5f95d985b74a84ca3455a7ae0a65b1e53e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-10T21:21:21.650 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:21 smithi050 bash[110145]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9 2024-05-10T21:21:21.913 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:21 smithi050 podman[110166]: 2024-05-10 21:21:21.653583514 +0000 UTC m=+0.648968743 container remove f8f98545bf4cf22a20915474eabad5f95d985b74a84ca3455a7ae0a65b1e53e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-10T21:21:22.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:22.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:22 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:22.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:22.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:22.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:22.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:22.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:22.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:22.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:22 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:22.327+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:22.794 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:22 smithi050 podman[110252]: 2024-05-10 21:21:22.358186007 +0000 UTC m=+0.038429430 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:21:22.795 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:22 smithi050 podman[110252]: 2024-05-10 21:21:22.47133656 +0000 UTC m=+0.151580036 container create 7006010ee2165e530a7f5a6b7cc2c226e619364e45b47ce4a61ea35a23929f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:21:22.795 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:22 smithi050 podman[110252]: 2024-05-10 21:21:22.79981753 +0000 UTC m=+0.480061022 container init 7006010ee2165e530a7f5a6b7cc2c226e619364e45b47ce4a61ea35a23929f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True) 2024-05-10T21:21:23.163 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:22 smithi050 podman[110252]: 2024-05-10 21:21:22.812412579 +0000 UTC m=+0.492656012 container start 7006010ee2165e530a7f5a6b7cc2c226e619364e45b47ce4a61ea35a23929f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-10T21:21:23.163 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:22 smithi050 podman[110252]: 2024-05-10 21:21:22.898518639 +0000 UTC m=+0.578762077 container attach 7006010ee2165e530a7f5a6b7cc2c226e619364e45b47ce4a61ea35a23929f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:23.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:23 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:23.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:23.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:23.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:23.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:23.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:23.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:23.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:23 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:23.315+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:23.663 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 10 21:21:23 smithi050 podman[110252]: 2024-05-10 21:21:23.289313816 +0000 UTC m=+0.969557275 container died 7006010ee2165e530a7f5a6b7cc2c226e619364e45b47ce4a61ea35a23929f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:21:24.036 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T21:21:24.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:24.125 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T21:21:24.125 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T21:21:24.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.10 2024-05-10T21:21:24.430 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.153+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 systemd[1]: Stopping Ceph osd.10 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.435+0000 7fcc9f573700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.435+0000 7fcc9f573700 -1 osd.10 87 *** Got signal Terminated *** 2024-05-10T21:21:24.431 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:24 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:24.435+0000 7fcc9f573700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:21:24.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:24.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:24.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:24.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:24.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:24.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:24 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:24.286+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:25.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:25.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:25 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:25.133+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:25.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:25.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:25.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:25.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:25.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:25.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:25.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:25.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:26.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:26 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:26.168+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:26.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:26.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:26 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:26.212+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:27.515 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:27 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:27.183+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.248+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.248+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.248+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.248+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:27.580 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:27 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:27.249+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:28.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:28.615 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:28.615 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:28 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:28.162+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:28.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:28.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:28.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:28.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:28.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:28.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:28.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:28 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:28.298+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:29.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:01.988273+0000 front 2024-05-10T21:20:01.988053+0000 (oldest deadline 2024-05-10T21:20:27.287887+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:12.589030+0000 front 2024-05-10T21:20:12.589057+0000 (oldest deadline 2024-05-10T21:20:36.689283+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:20.590977+0000 front 2024-05-10T21:20:20.591079+0000 (oldest deadline 2024-05-10T21:20:42.890936+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-10T21:20:26.392142+0000 front 2024-05-10T21:20:26.392449+0000 (oldest deadline 2024-05-10T21:20:52.292028+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:38.693466+0000 front 2024-05-10T21:20:38.693494+0000 (oldest deadline 2024-05-10T21:21:00.393488+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:45.094454+0000 front 2024-05-10T21:20:45.094534+0000 (oldest deadline 2024-05-10T21:21:07.994412+0000) 2024-05-10T21:21:29.614 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10[59450]: 2024-05-10T21:21:29.202+0000 7fcc97b21700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:51.396115+0000 front 2024-05-10T21:20:51.396349+0000 (oldest deadline 2024-05-10T21:21:17.295854+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:29.626 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:29 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:29.336+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:30.187 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:29 smithi092 podman[70898]: 2024-05-10 21:21:29.887727595 +0000 UTC m=+5.571684143 container died 7510d4927af1dcd707415a693c0ede6a58fb90eddd43f55caa91df727446acfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:21:30.455 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:30 smithi092 podman[70898]: 2024-05-10 21:21:30.193670114 +0000 UTC m=+5.877626652 container cleanup 7510d4927af1dcd707415a693c0ede6a58fb90eddd43f55caa91df727446acfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-10T21:21:30.455 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:30 smithi092 bash[70898]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10 2024-05-10T21:21:30.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:30.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:30 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:30.353+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:30.803 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:30 smithi092 podman[70927]: 2024-05-10 21:21:30.461701823 +0000 UTC m=+0.570917793 container remove 7510d4927af1dcd707415a693c0ede6a58fb90eddd43f55caa91df727446acfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-10T21:21:31.067 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:30 smithi092 podman[71017]: 2024-05-10 21:21:30.71167803 +0000 UTC m=+0.017457622 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:21:31.068 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:30 smithi092 podman[71017]: 2024-05-10 21:21:30.829524496 +0000 UTC m=+0.135304082 container create 12344a5150ae2ad0a0d4ed0f4f9ff1660a6f31e94b8b6712a0102df3ae611118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:31.068 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:31 smithi092 podman[71017]: 2024-05-10 21:21:31.07370892 +0000 UTC m=+0.379488509 container init 12344a5150ae2ad0a0d4ed0f4f9ff1660a6f31e94b8b6712a0102df3ae611118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-10T21:21:31.355 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:31 smithi092 podman[71017]: 2024-05-10 21:21:31.076879734 +0000 UTC m=+0.382659317 container start 12344a5150ae2ad0a0d4ed0f4f9ff1660a6f31e94b8b6712a0102df3ae611118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-10T21:21:31.355 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:31 smithi092 podman[71017]: 2024-05-10 21:21:31.165394842 +0000 UTC m=+0.471174433 container attach 12344a5150ae2ad0a0d4ed0f4f9ff1660a6f31e94b8b6712a0102df3ae611118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-10T21:21:31.613 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 10 21:21:31 smithi092 podman[71017]: 2024-05-10 21:21:31.361439082 +0000 UTC m=+0.667218669 container died 12344a5150ae2ad0a0d4ed0f4f9ff1660a6f31e94b8b6712a0102df3ae611118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:21:31.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:31.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:31.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:31.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:31.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:31.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:31.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:31 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:31.348+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:31.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T21:21:31.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:31.955 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T21:21:31.956 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T21:21:31.956 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.11 2024-05-10T21:21:32.241 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 systemd[1]: Stopping Ceph osd.11 for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:32.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.246+0000 7f7235aa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.246+0000 7f7235aa0700 -1 osd.11 87 *** Got signal Terminated *** 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.246+0000 7f7235aa0700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:32.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:32 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:32.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:33.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:33 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:33.362+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:34.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:34 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:34.359+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-05-10T21:21:08.213081+0000 front 2024-05-10T21:21:08.213043+0000 (oldest deadline 2024-05-10T21:21:34.013700+0000) 2024-05-10T21:21:35.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:35.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:35 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:35.310+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-05-10T21:21:08.213081+0000 front 2024-05-10T21:21:08.213043+0000 (oldest deadline 2024-05-10T21:21:34.013700+0000) 2024-05-10T21:21:36.624 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-10T21:20:00.602678+0000 front 2024-05-10T21:20:00.602422+0000 (oldest deadline 2024-05-10T21:20:26.502217+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-10T21:20:10.003412+0000 front 2024-05-10T21:20:10.003350+0000 (oldest deadline 2024-05-10T21:20:37.004058+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-05-10T21:20:21.605808+0000 front 2024-05-10T21:20:21.605910+0000 (oldest deadline 2024-05-10T21:20:43.305811+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-10T21:20:37.908740+0000 front 2024-05-10T21:20:37.908638+0000 (oldest deadline 2024-05-10T21:21:03.208425+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-05-10T21:20:46.709483+0000 front 2024-05-10T21:20:46.709382+0000 (oldest deadline 2024-05-10T21:21:07.809538+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-10T21:20:50.710736+0000 front 2024-05-10T21:20:50.710847+0000 (oldest deadline 2024-05-10T21:21:16.610500+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-10T21:21:02.412182+0000 front 2024-05-10T21:21:02.412235+0000 (oldest deadline 2024-05-10T21:21:24.712049+0000) 2024-05-10T21:21:36.625 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:36 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11[60117]: 2024-05-10T21:21:36.329+0000 7f722e04e700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-05-10T21:21:08.213081+0000 front 2024-05-10T21:21:08.213043+0000 (oldest deadline 2024-05-10T21:21:34.013700+0000) 2024-05-10T21:21:37.873 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:37 smithi142 podman[71655]: 2024-05-10 21:21:37.565743288 +0000 UTC m=+5.413317764 container died cb90a9a931c6706fc9e83b019fdfcc9cab94d50bb44e76eb248f0fc554da0ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2) 2024-05-10T21:21:37.873 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:37 smithi142 podman[71655]: 2024-05-10 21:21:37.879323732 +0000 UTC m=+5.726898205 container cleanup cb90a9a931c6706fc9e83b019fdfcc9cab94d50bb44e76eb248f0fc554da0ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-10T21:21:38.124 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:37 smithi142 bash[71655]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11 2024-05-10T21:21:38.124 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71674]: 2024-05-10 21:21:38.088807588 +0000 UTC m=+0.518583672 container remove cb90a9a931c6706fc9e83b019fdfcc9cab94d50bb44e76eb248f0fc554da0ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:38.779 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71767]: 2024-05-10 21:21:38.397834496 +0000 UTC m=+0.017939659 image pull d1bf1c877c43a8a2ce610d203b57dc6523d36919e23ee5048f81365034fece62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95 2024-05-10T21:21:38.780 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71767]: 2024-05-10 21:21:38.514503088 +0000 UTC m=+0.134608231 container create 6410bfa6e7e2fb2c4aebb1dce18414c8abd8ef74d78d614410c4bc88cccffdd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-10T21:21:38.780 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71767]: 2024-05-10 21:21:38.710608806 +0000 UTC m=+0.330713944 container init 6410bfa6e7e2fb2c4aebb1dce18414c8abd8ef74d78d614410c4bc88cccffdd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-10T21:21:38.780 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71767]: 2024-05-10 21:21:38.713577735 +0000 UTC m=+0.333682879 container start 6410bfa6e7e2fb2c4aebb1dce18414c8abd8ef74d78d614410c4bc88cccffdd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-10T21:21:38.780 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:38 smithi142 podman[71767]: 2024-05-10 21:21:38.785424289 +0000 UTC m=+0.405529435 container attach 6410bfa6e7e2fb2c4aebb1dce18414c8abd8ef74d78d614410c4bc88cccffdd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:21:39.124 INFO:journalctl@ceph.osd.11.smithi142.stdout:May 10 21:21:39 smithi142 podman[71767]: 2024-05-10 21:21:39.007145818 +0000 UTC m=+0.627250964 container died 6410bfa6e7e2fb2c4aebb1dce18414c8abd8ef74d78d614410c4bc88cccffdd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-10T21:21:39.613 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T21:21:39.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:39.649 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T21:21:39.649 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T21:21:39.649 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.a 2024-05-10T21:21:40.163 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:39 smithi050 systemd[1]: Stopping Ceph mds.a for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:40.663 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:21:40.277+0000 7fbe69c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:40.663 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:40 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a[69432]: 2024-05-10T21:21:40.277+0000 7fbe69c00700 -1 mds.a *** got signal Terminated *** 2024-05-10T21:21:44.604 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:44 smithi050 podman[110369]: 2024-05-10 21:21:44.303644514 +0000 UTC m=+4.175302262 container died 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-10T21:21:44.912 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:44 smithi050 podman[110369]: 2024-05-10 21:21:44.811818955 +0000 UTC m=+4.683476710 container cleanup 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:44.913 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:44 smithi050 bash[110369]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a 2024-05-10T21:21:45.409 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 10 21:21:45 smithi050 podman[110382]: 2024-05-10 21:21:45.146453098 +0000 UTC m=+0.821144006 container remove 28049d57f73fb0bdcd6c4260405baaa194c74aaa9c1c06a25d387e5a26eead91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:21:45.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T21:21:45.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:45.568 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T21:21:45.568 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T21:21:45.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.d 2024-05-10T21:21:45.913 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:45 smithi050 systemd[1]: Stopping Ceph mds.d for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:46.413 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d[68257]: 2024-05-10T21:21:46.147+0000 7ff15fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:46.413 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:46 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d[68257]: 2024-05-10T21:21:46.147+0000 7ff15fe00700 -1 mds.d *** got signal Terminated *** 2024-05-10T21:21:50.474 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:50 smithi050 podman[110489]: 2024-05-10 21:21:50.173299596 +0000 UTC m=+4.155372406 container died 16828d83ab51adc935494952d9776fc45bfb3c13b33357ffa532ecd35e90b8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-10T21:21:50.861 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:50 smithi050 podman[110489]: 2024-05-10 21:21:50.590652446 +0000 UTC m=+4.572725262 container cleanup 16828d83ab51adc935494952d9776fc45bfb3c13b33357ffa532ecd35e90b8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:21:50.862 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:50 smithi050 bash[110489]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d 2024-05-10T21:21:51.162 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 10 21:21:50 smithi050 podman[110504]: 2024-05-10 21:21:50.866779746 +0000 UTC m=+0.674016896 container remove 16828d83ab51adc935494952d9776fc45bfb3c13b33357ffa532ecd35e90b8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-10T21:21:51.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T21:21:51.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:51.488 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T21:21:51.488 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T21:21:51.488 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.g 2024-05-10T21:21:51.913 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:51 smithi050 systemd[1]: Stopping Ceph mds.g for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:52.413 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g[69029]: 2024-05-10T21:21:52.126+0000 7fadc7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:52.413 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:52 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g[69029]: 2024-05-10T21:21:52.126+0000 7fadc7600700 -1 mds.g *** got signal Terminated *** 2024-05-10T21:21:53.618 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:53 smithi050 podman[110609]: 2024-05-10 21:21:53.336519185 +0000 UTC m=+1.345534964 container died 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:53.913 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:53 smithi050 podman[110609]: 2024-05-10 21:21:53.744641457 +0000 UTC m=+1.753657260 container cleanup 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:21:53.913 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:53 smithi050 bash[110609]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g 2024-05-10T21:21:54.413 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 10 21:21:53 smithi050 podman[110623]: 2024-05-10 21:21:53.965720582 +0000 UTC m=+0.611342548 container remove 68383664576f9e7a24c7855bfb813d5b7f726ccb1aab6bde9dd0a14defc5e3d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-10T21:21:54.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T21:21:54.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:54.647 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T21:21:54.647 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T21:21:54.647 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.j 2024-05-10T21:21:55.162 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:54 smithi050 systemd[1]: Stopping Ceph mds.j for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:55.663 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j[67837]: 2024-05-10T21:21:55.270+0000 7f50e9c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:55.663 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:55 smithi050 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j[67837]: 2024-05-10T21:21:55.270+0000 7f50e9c00700 -1 mds.j *** got signal Terminated *** 2024-05-10T21:21:57.623 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:57 smithi050 podman[110729]: 2024-05-10 21:21:57.233288381 +0000 UTC m=+2.103500375 container stop dfc2796d73f317d26bd3a5479d7deb18bb2c80021bca3a6dca4e64212a3cb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-10T21:21:57.623 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:57 smithi050 podman[110729]: 2024-05-10 21:21:57.263215343 +0000 UTC m=+2.133427302 container died dfc2796d73f317d26bd3a5479d7deb18bb2c80021bca3a6dca4e64212a3cb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.schema-version=1.0) 2024-05-10T21:21:57.912 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:57 smithi050 podman[110729]: 2024-05-10 21:21:57.748772918 +0000 UTC m=+2.618984915 container cleanup dfc2796d73f317d26bd3a5479d7deb18bb2c80021bca3a6dca4e64212a3cb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059) 2024-05-10T21:21:57.913 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:57 smithi050 bash[110729]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j 2024-05-10T21:21:58.413 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 10 21:21:58 smithi050 podman[110755]: 2024-05-10 21:21:58.024467589 +0000 UTC m=+0.772428476 container remove dfc2796d73f317d26bd3a5479d7deb18bb2c80021bca3a6dca4e64212a3cb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-10T21:21:58.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T21:21:58.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:21:58.685 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T21:21:58.685 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T21:21:58.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.b 2024-05-10T21:21:58.962 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:21:58 smithi092 systemd[1]: Stopping Ceph mds.b for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:21:59.356 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:21:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: 2024-05-10T21:21:58.967+0000 7fc8dd5fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:21:59.357 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:21:58 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b[64809]: 2024-05-10T21:21:58.967+0000 7fc8dd5fa700 -1 mds.b *** got signal Terminated *** 2024-05-10T21:22:00.363 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:22:00 smithi092 podman[71122]: 2024-05-10 21:22:00.300571788 +0000 UTC m=+1.422549497 container died f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-10T21:22:00.863 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:22:00 smithi092 podman[71122]: 2024-05-10 21:22:00.66177641 +0000 UTC m=+1.783754125 container cleanup f3862f24d6c0d1114087068a098c9907bda95cf384714a52a0e5075927dac26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T21:22:00.863 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 10 21:22:00 smithi092 bash[71122]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-b 2024-05-10T21:22:01.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T21:22:01.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:01.113 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T21:22:01.113 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T21:22:01.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.e 2024-05-10T21:22:01.613 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:01 smithi092 systemd[1]: Stopping Ceph mds.e for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:01.613 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e[64183]: 2024-05-10T21:22:01.440+0000 7fe4184e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:01.614 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:01 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e[64183]: 2024-05-10T21:22:01.440+0000 7fe4184e5700 -1 mds.e *** got signal Terminated *** 2024-05-10T21:22:04.593 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:04 smithi092 podman[71254]: 2024-05-10 21:22:04.28873788 +0000 UTC m=+2.950938884 container stop fb2bdcff094ca731cac0dd5265e10651d23c12d93cb68945d4d67a1353584426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-10T21:22:04.593 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:04 smithi092 podman[71254]: 2024-05-10 21:22:04.314341368 +0000 UTC m=+2.976542373 container died fb2bdcff094ca731cac0dd5265e10651d23c12d93cb68945d4d67a1353584426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-10T21:22:04.863 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:04 smithi092 podman[71254]: 2024-05-10 21:22:04.681875592 +0000 UTC m=+3.344076597 container cleanup fb2bdcff094ca731cac0dd5265e10651d23c12d93cb68945d4d67a1353584426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-05-10T21:22:04.864 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:04 smithi092 bash[71254]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e 2024-05-10T21:22:04.864 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 10 21:22:04 smithi092 podman[71268]: 2024-05-10 21:22:04.849430917 +0000 UTC m=+0.554113654 container remove fb2bdcff094ca731cac0dd5265e10651d23c12d93cb68945d4d67a1353584426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:22:05.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T21:22:05.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:05.124 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T21:22:05.124 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T21:22:05.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.h 2024-05-10T21:22:05.363 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:05 smithi092 systemd[1]: Stopping Ceph mds.h for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:05.863 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: 2024-05-10T21:22:05.476+0000 7f79c8137700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:05.863 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:05 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h[64501]: 2024-05-10T21:22:05.476+0000 7f79c8137700 -1 mds.h *** got signal Terminated *** 2024-05-10T21:22:08.589 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:08 smithi092 podman[71374]: 2024-05-10 21:22:08.304007509 +0000 UTC m=+2.956893046 container died 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-10T21:22:08.863 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:08 smithi092 podman[71374]: 2024-05-10 21:22:08.677738698 +0000 UTC m=+3.330624243 container cleanup 6faac74d2d7354e90508277ebdda4746bfa5d7c0e1ba4a4f0e68e4e052c1e905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-10T21:22:08.863 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 10 21:22:08 smithi092 bash[71374]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-h 2024-05-10T21:22:09.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T21:22:09.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:09.118 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T21:22:09.118 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T21:22:09.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.k 2024-05-10T21:22:09.363 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:09 smithi092 systemd[1]: Stopping Ceph mds.k for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:09.863 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k[63878]: 2024-05-10T21:22:09.464+0000 7fec7995c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:09.863 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:09 smithi092 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k[63878]: 2024-05-10T21:22:09.464+0000 7fec7995c700 -1 mds.k *** got signal Terminated *** 2024-05-10T21:22:11.863 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:11 smithi092 podman[71494]: 2024-05-10 21:22:11.617052023 +0000 UTC m=+2.281515353 container died d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T21:22:12.194 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:11 smithi092 podman[71494]: 2024-05-10 21:22:11.990723146 +0000 UTC m=+2.655186479 container cleanup d294a4f8d7d722ea9d81aa657007089c347c745d803694595385028baa1d4627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-10T21:22:12.194 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 10 21:22:11 smithi092 bash[71494]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-k 2024-05-10T21:22:12.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T21:22:12.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:12.351 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T21:22:12.351 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T21:22:12.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.c 2024-05-10T21:22:12.648 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:12 smithi142 systemd[1]: Stopping Ceph mds.c for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:12.913 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: 2024-05-10T21:22:12.653+0000 7f6edbbe8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:12.913 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:12 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c[65585]: 2024-05-10T21:22:12.653+0000 7f6edbbe8700 -1 mds.c *** got signal Terminated *** 2024-05-10T21:22:16.549 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:16 smithi142 podman[71874]: 2024-05-10 21:22:16.293673078 +0000 UTC m=+3.741195426 container died 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:22:16.874 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:16 smithi142 podman[71874]: 2024-05-10 21:22:16.645306324 +0000 UTC m=+4.092828671 container cleanup 23ac4e1a9133478d3e6d0bfd59d9c44ae13dcc35242a233852d9cf04a47e6507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T21:22:16.874 INFO:journalctl@ceph.mds.c.smithi142.stdout:May 10 21:22:16 smithi142 bash[71874]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-c 2024-05-10T21:22:17.016 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T21:22:17.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:17.051 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T21:22:17.051 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T21:22:17.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.f 2024-05-10T21:22:17.360 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:17 smithi142 systemd[1]: Stopping Ceph mds.f for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:17.624 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f[65273]: 2024-05-10T21:22:17.365+0000 7f7f17abd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:17.624 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:17 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f[65273]: 2024-05-10T21:22:17.365+0000 7f7f17abd700 -1 mds.f *** got signal Terminated *** 2024-05-10T21:22:20.534 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:20 smithi142 podman[72008]: 2024-05-10 21:22:20.214749381 +0000 UTC m=+2.943038117 container died d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-10T21:22:20.534 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:20 smithi142 podman[72008]: 2024-05-10 21:22:20.540243365 +0000 UTC m=+3.268532101 container cleanup d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-10T21:22:20.874 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:20 smithi142 bash[72008]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f 2024-05-10T21:22:20.874 INFO:journalctl@ceph.mds.f.smithi142.stdout:May 10 21:22:20 smithi142 podman[72022]: 2024-05-10 21:22:20.716717956 +0000 UTC m=+0.500066714 container remove d10ac3efcd791ac358a95a97db3a9323aff8e2570d79be202d6760682bbc1f18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T21:22:20.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T21:22:20.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:20.991 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T21:22:20.991 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T21:22:20.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.i 2024-05-10T21:22:21.296 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:21 smithi142 systemd[1]: Stopping Ceph mds.i for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:21.624 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: 2024-05-10T21:22:21.302+0000 7fa0d07f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:21.624 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:21 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i[65892]: 2024-05-10T21:22:21.302+0000 7fa0d07f7700 -1 mds.i *** got signal Terminated *** 2024-05-10T21:22:24.546 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:24 smithi142 podman[72128]: 2024-05-10 21:22:24.293142854 +0000 UTC m=+3.085635608 container died 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-10T21:22:24.874 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:24 smithi142 podman[72128]: 2024-05-10 21:22:24.643782807 +0000 UTC m=+3.436275568 container cleanup 9afff81020a45df75b0f9e1bd4e1b82a93a0c5b6e2b5685b07b6a5864c38ce7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-10T21:22:24.874 INFO:journalctl@ceph.mds.i.smithi142.stdout:May 10 21:22:24 smithi142 bash[72128]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-i 2024-05-10T21:22:25.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T21:22:25.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:25.081 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T21:22:25.081 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T21:22:25.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.l 2024-05-10T21:22:25.624 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:25 smithi142 systemd[1]: Stopping Ceph mds.l for 55ba4db6-0f0f-11ef-bc98-c7b262605968... 2024-05-10T21:22:25.624 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l[64956]: 2024-05-10T21:22:25.397+0000 7fa989929700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T21:22:25.624 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:25 smithi142 ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l[64956]: 2024-05-10T21:22:25.397+0000 7fa989929700 -1 mds.l *** got signal Terminated *** 2024-05-10T21:22:29.116 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:28 smithi142 podman[72251]: 2024-05-10 21:22:28.86246275 +0000 UTC m=+3.556238114 container died f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T21:22:29.374 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:29 smithi142 podman[72251]: 2024-05-10 21:22:29.230697756 +0000 UTC m=+3.924473125 container cleanup f21d04795d09c40c8920ed798496c602d5c03e6ca63330b9c60a2cb16eefae01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:81e2576b298680f3bcbcb925e94a74aa6d04a55d005c1817cd773ad4498b3b95, name=ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.114851-debug-7b71059, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-10T21:22:29.374 INFO:journalctl@ceph.mds.l.smithi142.stdout:May 10 21:22:29 smithi142 bash[72251]: ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968-mds-l 2024-05-10T21:22:29.594 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55ba4db6-0f0f-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T21:22:29.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T21:22:29.624 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T21:22:29.624 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T21:22:29.962 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:22:35.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T21:22:35.321 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:22:37.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T21:22:37.967 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:22:40.240 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:22:40.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:22:40.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T21:22:40.395 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T21:22:40.397 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi050/crash 2024-05-10T21:22:40.398 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash -- . 2024-05-10T21:22:40.489 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T21:22:40.490 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-05-10T21:22:40.496 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi092/crash 2024-05-10T21:22:40.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash -- . 2024-05-10T21:22:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T21:22:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-05-10T21:22:40.547 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi142/crash 2024-05-10T21:22:40.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash -- . 2024-05-10T21:22:40.584 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T21:22:40.585 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-05-10T21:22:40.585 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T21:22:40.586 DEBUG:teuthology.orchestra.run.smithi050:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T21:22:40.693 INFO:teuthology.orchestra.run.smithi050.stdout:2024-05-10T21:16:34.928983+0000 mds.i (mds.2) 305 : cluster [WRN] Scrub error on inode 0x20000001518 (/volumes/qa/sv_1/0586cbad-ebd8-4ad1-95e3-90eeacc42f3f/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-40) see mds.i log and `damage ls` output for details 2024-05-10T21:22:40.698 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-10T21:22:40.698 INFO:tasks.cephadm:Compressing logs... 2024-05-10T21:22:40.698 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T21:22:40.703 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T21:22:40.705 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T21:22:40.729 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T21:22:40.729 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2024-05-10T21:22:40.729 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log 2024-05-10T21:22:40.729 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-10T21:22:40.730 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-10T21:22:40.730 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/cephadm.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-10T21:22:40.737 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi092.stderr: 89.9% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.2.log: /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.f.log: /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log: gzip 92.4% -5 -- replaced with /var/log/ceph/cephadm.log.gz --verbose 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr: -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.5.log: /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-10T21:22:40.738 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-10T21:22:40.739 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.z.log: 90.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T21:22:40.739 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T21:22:40.739 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.i.log: 89.9% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T21:22:40.739 INFO:teuthology.orchestra.run.smithi142.stderr: 94.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-10T21:22:40.739 INFO:teuthology.orchestra.run.smithi092.stderr: 92.9% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-10T21:22:40.740 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-10T21:22:40.740 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-10T21:22:40.740 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-10T21:22:40.741 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T21:22:40.741 INFO:teuthology.orchestra.run.smithi142.stderr: 82.1% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T21:22:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T21:22:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-10T21:22:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T21:22:40.747 INFO:teuthology.orchestra.run.smithi092.stderr: 82.1% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T21:22:40.753 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-10T21:22:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-10T21:22:40.760 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T21:22:40.761 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T21:22:40.762 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log 2024-05-10T21:22:40.763 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T21:22:40.764 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-10T21:22:40.765 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-10T21:22:40.767 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-10T21:22:40.767 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T21:22:40.768 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip 89.9% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T21:22:40.768 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-10T21:22:40.768 INFO:teuthology.orchestra.run.smithi050.stderr: 90.2% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T21:22:40.769 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-10T21:22:40.770 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-10T21:22:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-10T21:22:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr: 94.5% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr: 94.6% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T21:22:40.775 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-10T21:22:40.776 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T21:22:40.776 INFO:teuthology.orchestra.run.smithi050.stderr: 82.6% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T21:22:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-10T21:22:40.780 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.3% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-10T21:22:40.781 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-10T21:22:40.785 INFO:teuthology.orchestra.run.smithi142.stderr: 94.6% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-10T21:22:40.792 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-10T21:22:40.795 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T21:22:41.109 INFO:teuthology.orchestra.run.smithi050.stderr: 90.2% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-10T21:22:41.300 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-10T21:22:41.317 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-10T21:22:42.207 INFO:teuthology.orchestra.run.smithi050.stderr: 90.2% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-10T21:22:43.895 INFO:teuthology.orchestra.run.smithi142.stderr: 93.8% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-10T21:22:43.973 INFO:teuthology.orchestra.run.smithi092.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-10T21:22:44.107 INFO:teuthology.orchestra.run.smithi092.stderr: 95.2% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-10T21:22:44.190 INFO:teuthology.orchestra.run.smithi092.stderr: 95.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-10T21:22:44.865 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-10T21:22:44.895 INFO:teuthology.orchestra.run.smithi050.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-10T21:22:44.941 INFO:teuthology.orchestra.run.smithi092.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-10T21:22:45.044 INFO:teuthology.orchestra.run.smithi050.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-10T21:22:45.049 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-10T21:22:45.076 INFO:teuthology.orchestra.run.smithi050.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-10T21:22:45.097 INFO:teuthology.orchestra.run.smithi142.stderr: 93.8% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-10T21:22:45.134 INFO:teuthology.orchestra.run.smithi142.stderr: 95.4% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-10T21:22:45.162 INFO:teuthology.orchestra.run.smithi092.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-10T21:22:45.510 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-10T21:22:45.767 INFO:teuthology.orchestra.run.smithi050.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-10T21:23:17.248 INFO:teuthology.orchestra.run.smithi092.stderr: 94.0% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-10T21:23:17.249 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-05-10T21:23:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m36.530s 2024-05-10T21:23:17.250 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m58.496s 2024-05-10T21:23:17.250 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.942s 2024-05-10T21:23:19.485 INFO:teuthology.orchestra.run.smithi142.stderr: 94.1% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-10T21:23:19.487 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-05-10T21:23:19.487 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m38.767s 2024-05-10T21:23:19.487 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m57.806s 2024-05-10T21:23:19.487 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m1.832s 2024-05-10T21:24:17.787 INFO:teuthology.orchestra.run.smithi050.stderr: 93.7% -- replaced with /var/log/ceph/55ba4db6-0f0f-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-10T21:24:17.791 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-10T21:24:17.791 INFO:teuthology.orchestra.run.smithi050.stderr:real 1m37.056s 2024-05-10T21:24:17.791 INFO:teuthology.orchestra.run.smithi050.stderr:user 1m48.464s 2024-05-10T21:24:17.791 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m8.098s 2024-05-10T21:24:17.793 INFO:tasks.cephadm:Archiving logs... 2024-05-10T21:24:17.793 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi050/log 2024-05-10T21:24:17.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T21:24:33.304 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi092/log 2024-05-10T21:24:33.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T21:24:41.083 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi142/log 2024-05-10T21:24:41.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T21:24:49.020 INFO:tasks.cephadm:Removing cluster... 2024-05-10T21:24:49.021 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force 2024-05-10T21:24:49.354 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:24:50.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force 2024-05-10T21:24:50.515 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:24:50.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55ba4db6-0f0f-11ef-bc98-c7b262605968 --force 2024-05-10T21:24:51.018 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 55ba4db6-0f0f-11ef-bc98-c7b262605968 2024-05-10T21:24:51.371 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T21:24:51.371 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T21:24:51.433 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T21:24:51.449 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T21:24:51.465 INFO:tasks.cephadm:Teardown complete 2024-05-10T21:24:51.465 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T21:24:51.477 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T21:24:51.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T21:24:51.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T21:24:51.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T21:24:51.644 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T21:24:52.251 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T21:24:52.251 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T21:24:52.261 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T21:24:52.264 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T21:24:52.277 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-10T21:24:52.278 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-05-10T21:24:52.282 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T21:24:52.282 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-10T21:24:52.282 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +1557us[+1557us] +/- 51ms 2024-05-10T21:24:52.283 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 94 -1851us[-1849us] +/- 53ms 2024-05-10T21:24:52.283 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +1072ns[ +970ns] +/- 72ms 2024-05-10T21:24:52.283 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1622us[+1625us] +/- 51ms 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1758us[-1758us] +/- 54ms 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +77us[ +80us] +/- 72ms 2024-05-10T21:24:52.284 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T21:24:52.310 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-10T21:24:52.321 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T21:24:52.321 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-10T21:24:52.321 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1657us[+1687us] +/- 51ms 2024-05-10T21:24:52.322 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1774us[-1774us] +/- 54ms 2024-05-10T21:24:52.322 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +89us[ +89us] +/- 72ms 2024-05-10T21:24:52.322 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T21:24:52.323 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T21:24:52.338 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T21:24:52.338 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T21:24:52.357 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T21:24:52.595 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T21:24:52.665 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T21:24:52.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T21:24:52.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:52.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T21:24:52.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:52.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T21:24:52.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:52.893 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T21:24:52.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T21:24:52.918 INFO:teuthology.task.internal:Duration was 2387.897139 seconds 2024-05-10T21:24:52.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T21:24:52.932 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T21:24:52.932 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T21:24:52.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T21:24:52.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T21:24:52.980 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T21:24:52.981 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T21:24:53.090 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T21:24:53.406 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T21:24:53.407 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-05-10T21:24:53.407 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T21:24:53.475 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-05-10T21:24:53.475 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T21:24:53.501 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-05-10T21:24:53.501 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T21:24:53.536 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T21:24:53.536 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T21:24:53.540 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T21:24:53.544 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T21:24:53.602 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T21:24:53.602 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T21:24:53.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T21:24:53.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T21:24:54.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T21:24:54.792 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T21:24:54.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T21:24:54.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T21:24:54.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T21:24:54.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T21:24:54.930 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:54.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:54.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:54.964 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-05-10T21:24:54.984 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-05-10T21:24:55.031 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-05-10T21:24:55.072 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:55.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:55.121 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:55.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:55.136 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T21:24:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T21:24:55.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T21:24:55.164 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T21:24:55.165 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi050 2024-05-10T21:24:55.165 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T21:24:55.290 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi092 2024-05-10T21:24:55.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T21:24:55.327 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344/remote/smithi142 2024-05-10T21:24:55.328 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T21:24:55.365 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T21:24:55.365 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T21:24:55.373 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T21:24:55.375 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T21:24:55.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T21:24:55.441 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T21:24:55.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T21:24:55.453 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T21:24:55.453 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T21:24:55.461 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T21:24:55.466 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T21:24:55.476 INFO:teuthology.orchestra.run.smithi092.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 21:24 /home/ubuntu/cephtest 2024-05-10T21:24:55.481 INFO:teuthology.orchestra.run.smithi142.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 21:24 /home/ubuntu/cephtest 2024-05-10T21:24:55.512 INFO:teuthology.orchestra.run.smithi050.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 21:24 /home/ubuntu/cephtest 2024-05-10T21:24:55.516 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T21:24:55.529 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T21:24:55.568 INFO:teuthology.nuke:Checking targets against current locks 2024-05-10T21:24:55.596 DEBUG:teuthology.nuke:shortname: smithi050 2024-05-10T21:24:55.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T21:24:55.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.598082', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKAu78QdBJQI4w+tcTCL0d1bjZHGzNS50g5kx2KBlyg8TK+G0u/Pw6hFBFp5E/iYH87HbHv4ENdYHxGRxpz+VeI='} 2024-05-10T21:24:55.625 DEBUG:teuthology.nuke:shortname: smithi092 2024-05-10T21:24:55.625 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T21:24:55.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.597126', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHrcME+lzeCYOa/XtrRHQXNNS8PdEIgWMGVyxcYoXII00Vk+jDM2dmFb0lYaZNsImSm5cGbWSH4p66sBLNYVilc='} 2024-05-10T21:24:55.650 DEBUG:teuthology.nuke:shortname: smithi142 2024-05-10T21:24:55.650 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T21:24:55.666 INFO:teuthology.orchestra.console.smithi050:Power off 2024-05-10T21:24:55.666 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T21:24:55.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-10_15:47:15-fs-wip-vshankar-testing-20240509.114851-debug-testing-default-smithi/7701344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 20:35:59.598859', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0ULr/U/puljGF5t830NJviseco3C4f1/FfF0S1C8RBn/cvY7MLBqLD0njoYk49fStnogMv3c3MnKYvXVKxmv4='} 2024-05-10T21:24:55.689 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-05-10T21:24:55.690 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:55.710 INFO:teuthology.orchestra.console.smithi092:Power off 2024-05-10T21:24:55.710 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T21:24:55.723 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-05-10T21:24:55.733 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-05-10T21:24:55.733 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:55.749 INFO:teuthology.orchestra.console.smithi142:Power off 2024-05-10T21:24:55.750 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T21:24:55.759 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-10T21:24:55.772 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-05-10T21:24:55.772 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:55.797 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-05-10T21:24:59.724 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:59.760 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:59.799 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:24:59.836 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-05-10T21:24:59.873 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-10T21:24:59.909 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-05-10T21:25:03.838 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:25:03.874 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:25:03.911 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:25:03.950 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-05-10T21:25:03.985 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-05-10T21:25:04.022 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-05-10T21:25:04.051 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-05-10T21:25:07.986 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:25:08.022 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T21:25:08.098 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-05-10T21:25:08.133 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-05-10T21:25:08.199 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-05-10T21:25:08.234 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-05-10T21:25:08.340 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2387.8971388339996 failure_reason: 'error during quiesce thrashing: Error releasing set ''f0125e47'': 1 (EPERM)' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-10T21:25:08.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T21:25:08.429 INFO:teuthology.run:FAIL